UNIX Internals
Rohit Jnagal
[email protected]
Introduction
First Steps
Course Prerequisites
Prerequisites for the course:
C programming concepts and principles
Basic Computer Architecture concepts (refresher today!)
Basic algorithms and data structure concepts
Exposure to UNIX/Linux programming environment
Would be great if you also have:
Worked on a non-windows system
Paid attention in your earlier Operating System courses
Lots of Questions
Ability to stay awake through a 2 hours class
Operating System Refresher
Why an Operating System:
you dont want to know what goes into making pepperoni
Process & Resources (memory, disk, cpu, network)
Operating System Goals:
Create Resource Abstractions
Create Process Abstractions
Manage Resource Sharing
Process Isolation
Others ?
Operating System Refresher
Computer System Components:
Hardware
provides basic computing resources (CPU, Memory, I/O)
Operating System
controls and coordinates the use of hardware among various
application programs for various users.
Application Programs (Processes)
ways in which system used to solve problems of a user (compilers,
database systems, video games and making tea)
Users
(people, machines, other computers)
Operating Systems Refresher
Why learn Operating Systems:
To clear this course!
Easier to see how to use the system
To fine-tune performance of our workload
To add, modify or enhance a functionality in a transparent
way
To get new hardware up and running
To design OS-aware hardware and peripherals
OS is pure overload of real work. Applications are the ones
with real value.
Operating System History
Operating Systems through the ages:
Dark Ages ( 1945 55 )
Vacuum Tubes & Plugboards: the computing monstrosities
No OS
No Programming Languages
No Assembly Languages
Operating System History
Operating Systems through the ages:
1st Generation (1955 65 )
Mainframe Systems
Batching similar jobs
One job at a time
Automatic job sequencing
Resident Monitor
Initiate control in monitor
Control transfer to a job
When job completes, transfer back to
monitor
Severe I/O bottleneck
Operating System History
Operating Systems through the ages:
2nd Generation ( 1965 80 )
Integrated ICs Renaissance
Multi-programmed Batch Systems
Notion of computer architecture
Disk drives
On-line terminals
Several jobs in memory at the same
time
OS features
Job scheduling
Memory management ( Overlays)
CPU scheduling
Protection
Operating System History
Operating Systems through the ages:
2nd Generation (1965 80 )
Time-Sharing Systems
OS features:
Swapping
Virtual Memory
File System
CPU scheduling
Device Allocation
IPC
Interactive shell
Protection
Operating System History
A Long Time Ago, in a galaxy far, far away:
OS/360 Multiprogramming
MIT CTSS (Compatible Time Sharing)
MIT, Bell Labs, GE, MULTICS (Multiplexed Information and
Computing Service)
A game called Space Travel
And UNIX was born in 1969
Operating System History
Operating Systems through the ages:
3rd Generation (1980 - )
LSIs, VLSIs, Nano mobile an ant can use
Architecture Advances:
Smaller, faster Microprocessors
Caches
Larger Faster Storage
Offloading CPU work
Modern OS feature
GUI
Multimedia
Internet & Web
Networked, Distributed, etc
Operating System History
Current UNIX Systems:
HP HP-UX
HP ( Digital (Compaq) ) Tru64
Linux ( not exactly UNIX!)
FreeBSD, NetBSD
SUN Solaris
IBM AIX
Apple Mac OS X, etc
NSK (what about Non-Stop)
Operating System Classifications
Multi-user / Single user
Multiprogramming
Multi-threaded
User threads / thread libraries
System/Kernel threads
Monolithic Kernel / Micro-kernels
Functional Classification
Mainframe
Server
Desktop
Topographic Classification
Multiprocessor
Clustered Systems
Distributed Systems
Requirements/Constraints Classification
Embedded Systems
Real-Time Systems
UNIX Operating System
UNIX Structure
applications
Memory
Mgmt
MP
I/ O
Proc/
thread
mgmt
IPC
hardware
Misc...
LVM
File systems
shells
daemons
cmds
drivers
UNIX features
Files and Processes
File system has places ( / vs C:, D: )
Processes have life
Everything is a file
Processing environment
Hardware mode support
Interrupts and Exceptions
Processor Execution Levels
Process Management Subsystem
Memory Management Subsystem
Kernel/System Interfaces
So, whats different in a Non-UNIX system ?
Processor Modes
Supervisor or User Mode
Supervisor Mode
Can execute all instructions
Can reference all memory locations
Kernel execute in this mode
User Mode
Can execute subset of instructions
Can reference a subset of memory locations
To go from user to supervisor mode:
Interrupts/traps
Requests (system calls)
System Hierarchies
User program
User Level
Libraries
Systemcall interface
Kernel Level
Virtual File System
Phys File
System
Buffer Cache
LVM
device drivers
Hardware Level
lightweight system calls
Process Management
Subsystem
Memory
Management
Subsystem
IPC
Scheduling
I/ O
Hardware control
Primary Subsystems
Process Management
Manages all active processes and threads
Memory Management
Allocates memory for processes at run-time
Performs translations of physical memory locations to virtual memory
locations
File Subsystem
Virtual File System Layer
Abstracts and manages data for different file system types
Physical File System Layer
Manages access for specific file systems of various types
I/O Subsystem
Manages system I/O resources, drivers, and data structures
Drives system configuration process
Is my system UNIX or UNIX compliant ?
AT&T UNIX System V
Berkeley Software Distributions (BSD)
POSIX Standards
UNIX 2003 Standards
UNIX ABI
UNIX Kernel Interface / System Call Interface
HP-UX History
HPUX release History
Hewlett Packard UNIX
HPUX 1.0 in 1983
HPUX 10.0 (1995) : based on SVR4 UNIX standard
HPUX 11.0 (1997) : First 64-bit version
HPUX 11.11(2001) -> 11i series (World War 1 Armistice)
HPUX 11.20 (2002): First Itanium release
HPUX 11.23 11i v2 (2003): supports both PA and Itanium,
supports ccNUMA
HP-UX Specials
HP-UX features
First to use access control lists for file access permissions
First UNIX system with built in Volume Manager
Enhanced Reliability, security and partitioning
Clustering technology
Cell Based, ccNUMA architecture support
Virtual Server Environment (VSE)
Scaling
128 processors (expandable to 256)
1 Tb main memory
32 Tb maximum filesystem size
2 Tb maximum file size
Learning
check
Computer Architecture Refresher
Components:
CPU
Instruction Set Architecture
RISC Vs CISC
x86 CISC -> RISC +
micro-ops
IA32, IA64, PA-RISC, PowerPC, ARM
EPIC: Itanium
Pipelining
Fetch, Decode, Execute
Superpipelined, Superscalar
Registers
General Purpose Registers
Program Counters
Program Status Word (PSW)
Memory & Memory Hierarchy
Caches
RAM
Disks
Tapes
Peripherals
Computer Architecture Refresher
OS and Architecture :
Mutual Interaction
OS functionality limited by Architecture features
Multiprogramming on 8086
OS structure can be simplified with Architecture support
Interrupt, DMA, etc
Most proprietary OS are developed for a particular architecture
PA-RISC , IPF
x86
SPARC
Computer Architecture Refresher
Architecture support for OS:
Interrupt and Exceptions
How does the kernel notice an I/O has finished
Polling
Hardware Interrupt
Interrupts
Generated by Hardware Devices
Asynchronous
Exceptions
Generated by executing instructions
Synchronous
Trap (expected) or fault (unexpected)
Interrupt/Exception Handling
Computer Architecture Refresher
Architecture support for OS:
I/O Operations
Data Transfer Modes in I/O
Programmed I/O
DMA (Direct Memory Access)
Timers
How does OS take control of CPU from running programs?
Hardware timer
Timer Interrupt
Protected/Privileged Instructions
Direct I/O access
Memory State Management
Setting special mode bits
Halting a system
Computer Architecture Refresher
Architecture support for OS:
OS Protection
How does the processor know if a protected instruction should be
executed?
Rings or Modes of Operation
Access Rights
Crossing Protection Boundaries
User process call Kernel to do something privileged
OS defined sequence of System Calls
Modes, Contexts and Switches
Memory Protection
Protect user programs from each other
Protect itself from user programs
Memory Management Unit
Segmentation
Virtual Memory
Page table, Page protection, TLBs
Base and Limit Registers
Computer Architecture Refresher
Architecture support for OS:
Synchronization
Problems
Interrupt may occur at any time and may interfere with interrupted code
Synchronization of Concurrent Processes
Multi-processor synchronization
Solutions
Turn Off/On Interrupts
Use special atomic instructions
read-modify-write
test-and-set
Bus locks
Synchronization primitives spinlocks, semaphores, etc
Learning
check
System Calls
Methods to request system action on behalf of a user
Request
System Action
On behalf of a user
A Single program makes numerous system calls.
Program: Read from one file and write to another
System Calls
Passing Parameters
Three general methods:
Pass parameters in registers
Store them in memory and pass the address
Push the parameters on the stack
Usual rules apply: isolation, security, efficiency
System Calls
Types of System Calls
Process control
Create, execute, wait, end, abort, etc
File Management
Create, delete, open, close, read, write, copy, get/set attributes
ls, cat, more, grep etc.
Device Management
Read, write, attach, detach, get/set attributes
Information
Time, date, attributes
Communications
Send, receive, connect, accept, attributes
Making a System Call
Just use the stub!
Internally,
Stub pushes the syscall number on stack/register/u-area
Arguments go on stack/registers/u-area
A special trap instruction is called
Changes execution mode to kernel
Transfers control to syscall handler
using system call dispatch vector
Set return value and error status in register/locations
Restore hardware context
Returns to user mode and the stub
Learning
check
