Introduction To UNIX
Introduction To UNIX
UNIX is a popular and influential operating system that has played a significant role in the
development of modern computing. It was first developed in the 1960s and 1970s by a group of
researchers at Bell Labs, including Ken Thompson and Dennis Ritchie.
UNIX was designed to be a portable, multitasking, and multi-user operating system. It provides a set
of tools and utilities that enable users to interact with the computer system. Some key features of
UNIX include:
1. Multitasking: UNIX allows multiple tasks or processes to run concurrently, sharing system
resources such as the CPU and memory.
2. Multi-user: UNIX supports multiple users to simultaneously access the system and run their
processes. Each user has their own account and can have separate file spaces and
preferences.
3. Hierarchical file system: UNIX organizes files in a hierarchical directory structure, with
directories (folders) containing files and subdirectories. This structure allows for easy
organization and navigation of files.
4. Command-line interface: UNIX primarily uses a command-line interface (CLI) for interacting
with the system. Users enter commands through a terminal to perform various tasks, such
as running programs or manipulating files.
5. Shell scripting: UNIX provides a powerful scripting language called the shell. Shell scripts
allow users to automate repetitive tasks and create custom command sequences.
6. Portability: One of UNIX's original design goals was to be portable across different hardware
platforms. This portability has allowed UNIX to be adapted and used on a wide range of
systems, from mainframes to personal computers.
UNIX has had a significant influence on the development of other operating systems and has
inspired the design of modern operating systems, including Linux and macOS (which is based
on BSD, a UNIX-like system).
In UNIX, the operating system can be divided into two main components: the kernel and the
shell.
Kernel:
The kernel is the core of the UNIX operating system. It is responsible for managing system
resources, providing low-level services, and acting as an interface between the hardware
and the software running on the system. Some of the key functions performed by the kernel
include:
• Process management: The kernel manages processes, which are running instances
of programs. It schedules and controls the execution of processes, allocates system
resources to them, and facilitates inter-process communication.
• File system management: The kernel manages the file system, which includes
organizing and accessing files and directories on storage devices. It provides services
for creating, reading, writing, and deleting files, as well as managing permissions and
file metadata.
• Device management: The kernel interacts with hardware devices, such as disks,
network interfaces, and input/output devices. It provides device drivers that allow
higher-level software to communicate with and control these devices.
• System calls: The kernel exposes a set of system calls, which are functions that
applications can invoke to request services from the kernel. These system calls
provide an interface for applications to access kernel-level functionalities, such as
process management, file operations, and network communication.
Shell:
The shell is a command-line interface (CLI) through which users interact with the UNIX
system. It acts as a user-level program that interprets commands and executes them on
behalf of the user. The shell allows users to enter commands, run programs, manipulate
files, and perform various system operations.
Some commonly used shells in UNIX-like systems include:
• Bourne shell (sh): The original UNIX shell developed by Stephen Bourne. It provides a
basic set of features and syntax.
• C shell (csh): Developed by Bill Joy, the C shell introduced more interactive features,
such as command history and job control.
• Korn shell (ksh): The Korn shell, developed by David Korn, combines features from
the Bourne shell and the C shell, adding powerful scripting capabilities.
• Bash (Bourne Again SHell): Bash is a widely used shell that is compatible with the
Bourne shell and includes additional features. It is the default shell for many UNIX-
like systems, including Linux.
The shell interprets user commands, expands variables and wildcards, executes programs,
and manages input and output redirection. It also supports scripting, allowing users to write
shell scripts that automate tasks by combining multiple commands and control structures.
The shell interacts with the kernel by invoking system calls and receiving the results. It acts
as an intermediary, providing an interface between the user and the underlying operating
system.
In the UNIX operating system, files and processes are fundamental components that play
essential roles in system operation and user interactions.
Files:
In UNIX, a file is a named collection of data that is stored on a storage device, such as a hard
disk or solid-state drive. Files can represent various types of information, including text
documents, programs, configuration files, and directories (which are themselves a type of
file). Here are some key aspects related to files in UNIX:
Filesystem Hierarchy: UNIX organizes files in a hierarchical directory structure, with the root
directory ("/") at the top. Directories can contain files and subdirectories, forming a tree-like
structure. This organization allows for easy navigation and management of files.
• File Permissions: UNIX implements a robust permission system that specifies who
can read, write, or execute a file. Permissions are set for the file owner, group, and
others, providing control over file access and security.
• File Operations: Users can perform various operations on files, such as creating,
reading, writing, copying, moving, and deleting files. These operations can be
executed through command-line utilities or file managers in graphical environments.
• File Attributes: Each file in UNIX has associated metadata, including information
about file size, timestamps (such as the last modified time), ownership, permissions,
and file type.
Processes:
A process is an executing instance of a program within the UNIX system. It represents the
execution of a sequence of instructions, which can range from simple commands to complex
applications. Key points about processes in UNIX include:
Process Creation: Processes can be created by the operating system, by other processes, or
by users running commands. The operating system assigns a unique process identifier (PID)
to each process.
• Process States: Processes in UNIX can exist in different states, including running,
sleeping, stopped, and terminated. The operating system manages the transition of
processes between these states based on events and scheduling policies.
Processes interact with the kernel to request resources, access files, and perform system
operations. They can execute concurrently, allowing multiple processes to run
simultaneously, sharing system resources such as the CPU and memory.
Files and processes are closely linked in UNIX, as processes often operate on files, reading
from and writing to them or manipulating file-related metadata. The ability to manage and
manipulate files and processes efficiently is crucial for effective system administration and
user productivity in the UNIX environment.
System Calls
In UNIX-like operating systems, system calls are the primary interface between user-level
applications and the operating system kernel. System calls allow applications to request
services from the kernel, such as file operations, process management, network
communication, and hardware access. Here are some common system calls in UNIX:
Features of UNIX
UNIX, as an operating system, encompasses several features that have contributed to its popularity
and longevity. Here are some key features of UNIX:
• Portability: UNIX was designed with portability in mind. Its underlying principles and
architecture allowed it to be easily adapted to various hardware platforms, making it one of
the most portable operating systems. This portability has enabled UNIX to be used on a wide
range of devices, from mainframes to personal computers and embedded systems.
• Hierarchical File System: UNIX utilizes a hierarchical file system, organizing files and
directories in a tree-like structure. This hierarchical organization facilitates easy navigation
and management of files and directories.
• Shell Scripting: UNIX shells offer robust scripting capabilities, enabling users to write scripts
to automate tasks and create custom command sequences. Shell scripts can combine
multiple commands, control structures, and variables, providing a flexible and powerful
scripting environment.
• Pipes and Filters: UNIX introduced the concept of pipes and filters, which allow the output of
one command to be directly passed as input to another command. This mechanism enables
the composition of simple, reusable commands to perform complex operations and process
large amounts of data efficiently.
• Networking and Interprocess Communication: UNIX has built-in support for networking and
interprocess communication (IPC). It provides a wide range of protocols, tools, and libraries
for network communication, allowing UNIX systems to seamlessly connect and interact with
each other. Additionally, UNIX offers various IPC mechanisms, such as shared memory,
semaphores, and message queues, facilitating communication and synchronization between
processes.
• Modularity and Extensibility: UNIX follows a modular design, with small, independent
utilities that can be combined to accomplish more complex tasks. This design promotes
extensibility and encourages the development of new tools and applications that integrate
with existing UNIX systems.
• Security and Permissions: UNIX incorporates a robust security model with file permissions
and access controls. Each file has permissions that define who can read, write, and execute
it. User accounts and access privileges are managed by the operating system, ensuring data
security and preventing unauthorized access.
• Stability and Longevity: UNIX has a long history and has proven to be a stable and reliable
operating system. It has served as the foundation for numerous derivative systems, including
various commercial UNIX versions and open-source UNIX-like systems like Linux and
FreeBSD.
POSIX
POSIX (Portable Operating System Interface) is a set of standards that define a common interface for
operating systems. It aims to ensure compatibility and portability among different UNIX-like systems,
meaning that programs written for one POSIX-compliant system can be easily run on other POSIX-
compliant systems without needing major modifications.
POSIX standards specify various aspects of an operating system, including system calls, command-
line utilities, shell functionality, and more. By adhering to POSIX standards, operating systems can
provide a consistent and standardized environment for developers and users. This allows software
developers to write applications that can run on multiple POSIX-compliant systems, reducing the
need for rewriting or modifying code for each specific system.
In simple terms, POSIX helps make different UNIX-like operating systems work in a similar way,
ensuring that programs can run on multiple systems without much hassle. It promotes compatibility
and portability, making it easier for software developers to create cross-platform applications and
for users to work with different UNIX-like systems.
Single User Specification
In simple words, single user specification in UNIX refers to a mode or configuration where the
operating system allows only one user to access and use the system at a time. In this mode, the
system is designed to work with a single user, and other users are not allowed to log in or perform
any actions on the system simultaneously.
This mode is often used during system maintenance, troubleshooting, or other specific tasks where
it is necessary to limit access to the system to a single user. It provides a controlled environment and
helps prevent conflicts or issues that can arise from multiple users accessing and modifying the
system simultaneously.
When the system is in single user mode, it typically boots with minimal services and provides direct
access to the system's administrative functions. This allows the authorized user to perform tasks
such as system configuration, software installation, or system recovery without interference from
other users.
In summary, single user specification in UNIX refers to a configuration where only one user can
access and use the system at a time, providing a controlled environment for specific administrative
tasks or troubleshooting purposes.
Internal Commands:
Internal commands, also known as built-in commands, are commands that are directly implemented
within the shell (command-line interpreter). These commands are part of the shell's functionality
and are executed directly by the shell itself, without invoking an external program. Some examples
of internal commands in UNIX shells like Bash are:
Since internal commands are built into the shell, they typically execute faster than external
commands and have direct access to the shell's resources and variables.
External Commands:
External commands, as the name suggests, are commands that are implemented as separate
executable programs external to the shell. These commands are typically standalone programs
located in specific directories specified in the system's PATH variable. When an external command is
invoked, the shell locates the executable file and runs it. Examples of external commands in UNIX
include:
External commands are usually developed independently of the shell and can be written in various
programming languages. They are invoked by the shell through system calls and executed as
separate processes.
Utilities of UNIX
Calendar (cal), Display system date (date), Message display (echo), Calculator
(bc), Password changing (password), Knowing who are logged in (who),
Systeminformation using uname, File name of terminal connected to the
standard input (tty)
UNIX provides a wide range of utilities and commands to perform various tasks. Here are some
examples of utilities in UNIX, along with a brief description of their functionality:
• Calendar (cal): The cal utility displays a calendar for a specific month or year. It can show the
calendar in various formats, including monthly, yearly, or Julian calendar.
• Display system date (date): The date utility displays the current system date and time. It
allows customization of the date and time format and can also be used to set the system
date and time.
• Message display (echo): The echo utility prints a specified message or variable value to the
terminal.
• Calculator (bc): The bc utility is a basic calculator that allows mathematical calculations. It
supports various arithmetic operations, including addition, subtraction, multiplication,
division, and more.
• Password changing (passwd): The passwd utility is used to change a user's password. It
prompts the user to enter the old password and then allows the user to set a new password
following certain security requirements.
• Knowing who is logged in (who): The who utility displays a list of users currently logged into
the system. It shows the username, terminal or remote connection details, login time, and
other information about active user sessions.
• System information using uname: The uname utility provides information about the current
system. It can display details such as the operating system name, version, machine
architecture, and more.
• File name of the terminal connected to the standard input (tty): The tty utility displays the
file name of the terminal connected to the standard input. It is often used in shell scripts to
determine the current terminal device.
UNIX file system
In simple words, the UNIX file system is a way of organizing and storing files and directories in a
UNIX-like operating system. It follows a hierarchical structure starting from the root directory ("/")
and branching out into subdirectories and files.
• Files: In UNIX, everything is treated as a file. This includes regular files that contain data,
directories that store other files and directories, special files representing devices, and
symbolic links that point to other files or directories.
• File Naming: File names in UNIX are case-sensitive and can include alphanumeric characters
and certain special characters. However, some characters have special meanings and are
reserved.
• Directories: Directories are containers that hold files and other directories. They form a
parent-child relationship, creating a hierarchical structure. Each directory has a parent
directory, except for the root directory which is the top-level directory.
• Pathnames: Pathnames are used to specify the location of a file or directory in the file
system. An absolute pathname starts from the root directory, while a relative pathname is
relative to the current working directory.
• Dot (.) and Dot dot (..): In UNIX, the dot (.) represents the current directory, and the dotdot
(..) represents the parent directory.
• Commands: UNIX provides various commands to work with the file system. For example, "ls"
lists the contents of a directory, "cd" changes the current working directory, "mkdir" creates
a new directory, and "rm" removes files and directories.
Overall, the UNIX file system provides a structured way to organize and access files and directories.
It allows users to navigate the file system, create, modify, and delete files and directories, and
perform various operations on them.
The cd command is used to change the current working directory. You can provide the directory's
absolute or relative pathname as an argument to navigate to a different directory.
• Example:
• $ cd /home/user/documents # Changes to the /home/user/documents directory
• $ cd .. # Changes to the parent directory
• $ cd # Changes to the user's home directory
The mkdir command creates a new directory with the specified name. You can provide an absolute
or relative pathname to specify the location of the new directory.
• Example:
• $ mkdir new_directory # Creates a new directory called "new_directory" in the current
directory
• $ mkdir /path/to/new_directory # Creates a new directory with an absolute pathname
The rmdir command removes an empty directory. It can only delete directories that don't contain
any files or subdirectories.
• Example:
• $ rmdir empty_directory # Removes the "empty_directory" if it is empty
The ls command lists the contents of a directory. It displays the files and subdirectories within the
specified directory.
• Example:
• $ ls # Lists the contents of the current directory
• $ ls /path/to/directory # Lists the contents of the specified directory
These commands are basic and widely used in UNIX to navigate, create, remove directories, and
view the contents of directories.
In UNIX, there are three primary types of files: ordinary files, device files, and directory files. Here's a
brief explanation of each:
Ordinary Files:
• Ordinary files, also known as regular files, are the most common type of files in UNIX. They
store user data and can be text files, binary files, scripts, or any other type of file that
contains information. Ordinary files do not have any special attributes or characteristics
beyond the data they store.
Device Files:
• Device files represent hardware devices in the UNIX file system. They provide a way to
interact with hardware devices through the file system interface. Device files are further
classified into two types:
o Character Device Files: Character device files, also called character special files,
represent devices that operate on streams of characters. Examples include serial
ports, terminals, or sound cards. Character device files provide sequential input or
output access.
o Block Device Files: Block device files, also known as block special files, represent
devices that operate on fixed-size blocks of data. Examples include hard drives,
solid-state drives, or USB drives. Block device files allow random access to data,
reading and writing data in blocks.
Directory Files:
• Directory files are special files that act as containers for other files and directories. They
organize the hierarchical structure of the file system by providing a way to group related
files and directories together. Directory files store the names and metadata of the files and
subdirectories they contain. They allow users to navigate the file system hierarchy and
access the contents of the directories.
These three types of files (ordinary files, device files, and directory files) form the foundation of the
UNIX file system. They provide a flexible and organized way to store data, interact with hardware
devices, and navigate the file system hierarchy.
Ordinary files
Handling ordinary files in UNIX involves various operations such as displaying and creating files,
copying files, deleting files, renaming/moving files, paging output, printing files, determining file
types, counting lines/words/characters, comparing files, finding common elements between files,
and working with archive and compressed files. Here's a brief overview of these operations:
The cat command is used to display the contents of a file on the terminal. It can also be used to
create new files by combining or redirecting input from other sources.
The cp command is used to make a copy of a file. It can create a duplicate of an existing file with a
different name or copy a file to a different location.
Deleting a File (rm):
The rm command is used to delete or remove a file from the file system. Once a file is deleted with
rm, it cannot be easily recovered.
The mv command is used to rename or move a file. It can change the name of a file within the same
directory or move a file to a different directory.
The more command is used to display the contents of a file one page at a time on the terminal. It
allows scrolling through the file and navigating back and forth.
The lp command is used to send a file to a printer for printing. It enables users to print the contents
of a file on a connected printer.
The file command is used to determine the type of a file. It provides information about the file
format, such as whether it is a text file, binary file, or a specific file format.
The wc command is used to count the number of lines, words, and characters in a file. It provides a
summary of these counts when applied to a specific file.
The cmp command is used to compare two files byte by byte and determine whether they are
identical or differ at specific locations.
The comm command is used to find common lines between two sorted files. It can also display
unique lines present in each file.
The diff command is used to compare the contents of two files line by line and display the
differences between them.
The tar command is used to create an archive file that combines multiple files and directories into a
single file. It is often used for backup and compression purposes.
The gzip command is used to compress files, reducing their size. The gunzip command is used to
decompress gzip-compressed files.
Creating and Extracting Archive Files (zip and unzip): The zip command is used to create archive files
in ZIP format, which can contain multiple files and directories. The unzip command is used to extract
files from a ZIP archive.
File Attributes
In UNIX, each file has a set of attributes that define its properties and permissions. Here are the
important file attributes in UNIX:
1. File Name: The file name is the name assigned to a file. It is used to identify and reference
the file in the file system.
2. File Type: The file type indicates the nature or format of the file. It can be a regular file,
directory, symbolic link, device file, socket, or named pipe.
3. File Size: The file size represents the amount of storage space occupied by the file, typically
measured in bytes.
4. Permissions: UNIX uses a permission system to control file access. The permissions are set
for three categories of users: owner, group, and others. The permissions determine whether
a user can read, write, or execute the file.
5. Owner and Group: Each file has an owner assigned to it, who has certain privileges and
permissions over the file. The group attribute specifies the group to which the file belongs.
Access Time (atime): The last time the file was accessed or read.
Modification Time (mtime): The last time the file's content was modified.
Change Time (ctime): The last time the file's metadata (permissions, owner, etc.) was
modified.
7. Inode: An inode is a data structure that stores information about a file. It contains details
such as file size, ownership, permissions, timestamps, and pointers to the file's data blocks.
8. Links: UNIX allows multiple file names (hard links) to point to the same underlying file. The
link count attribute indicates the number of hard links associated with a file.
These file attributes provide information about the file's characteristics, permissions, ownership, and
metadata. They are used to manage file access, track file changes, and enforce file system security in
UNIX.
In UNIX, file permissions can be changed using two methods: relative permission and absolute
permission.
Relative Permission: Relative permission allows you to modify the existing permissions of a file by
adding or subtracting permissions. The most commonly used symbols are:
+ to add permissions.
- to remove permissions.
For example, to add read and write permissions for the owner of a file, you can use the command:
Absolute Permission: Absolute permission allows you to set the permissions explicitly using numeric
values. The permissions are represented by three digits, each digit representing the permissions for
the owner, group, and others, respectively. The numeric values for permissions are:
For example, to set read and write permissions for the owner, read-only permissions for the group,
and no permissions for others, you can use the command:
In UNIX, the ownership of a file can be changed using the chown command. Only the superuser or
the current owner of the file can change the ownership.
To change the ownership of a file, you need to specify the new owner using either their username or
user ID (UID). The basic syntax of the chown command is:
In UNIX, the group ownership of a file can be changed using the chgrp command. Similar to changing
ownership, only the superuser or the current owner of the file can change the group ownership.
To change the group ownership of a file, you need to specify the new group using either the group
name or group ID (GID). The basic syntax of the chgrp command is:
For example, to change the group ownership of a file to a group named "staff":
Inodes: An inode is a data structure that contains metadata about a file, such as its permissions,
ownership, timestamps, and physical location on the storage device. Inodes store information about
the file's attributes and the location of data blocks that hold the file's contents.
File System: The file system is responsible for managing the organization and storage of files on the
disk. It provides a way to access and manipulate files, directories, and their associated metadata.
UNIX supports various file systems, such as ext4, XFS, and ZFS, each with its own features and
capabilities.
The combination of the file system and inodes enables UNIX to efficiently manage and access files
and directories, allowing for reliable and secure file operations within the system.
Hard Link: A hard link is a direct reference to a file using its inode. Multiple hard links can point to
the same file, and all the links are treated equally. Deleting any hard link does not affect the other
links, as the file itself is not deleted until all hard links are removed.
Soft Link (Symbolic Link): A soft link, also known as a symbolic link or symlink, is a special type of file
that acts as a pointer to another file or directory. Unlike hard links, soft links are independent files
with their own inodes. If the target file or directory is moved or deleted, the soft link becomes
broken.
The file attribute for a directory is significant as it determines the permissions and access rights for
accessing and manipulating the contents of the directory. The directory attribute specifies whether a
user can list the directory's contents, create new files or directories within it, or modify existing files
and directories.
Default Permissions: When a new file or directory is created in UNIX, it inherits default permissions
that are set by the umask (user file-creation mask) value. The default permissions define the initial
access rights for the owner, group, and others. The default permissions can be modified by changing
the umask value.
Using umask: The umask command is used to set the default permissions for newly created files and
directories. The umask value is subtracted from the base permission (typically 666 for files and 777
for directories) to determine the effective permissions. The resulting permissions are then modified
based on the umask value.
With a umask value of 0022, the resulting permissions for newly created files are typically 644 (666 -
0022), which grants read and write permissions to the owner and read-only permissions to the
group and others. For directories, the resulting permissions are usually 755 (777 - 0022), providing
read, write, and execute permissions to the owner, and read and execute permissions to the group
and others.
Listing of Modification and Access Time: Modification Time (mtime): The modification time of a file
indicates the last time the file's content was modified or changed. It is updated whenever the file's
content is modified.
Access Time (atime): The access time of a file represents the last time the file was accessed or read.
It is updated whenever the file is accessed, regardless of whether the content is modified or not.
The touch command is used to change the timestamps of a file without modifying its content. It can
be used to update the access time (atime), modification time (mtime), or both. If the file does not
exist, touch creates an empty file with the specified timestamps.
The find command is used to locate files and directories based on specific criteria, such as file name,
size, type, modification time, and ownership. It searches a given directory and its subdirectories to
find matching files or directories and performs actions on them, such as displaying their paths or
executing commands on them.
Shell
The shell is a command-line interface that acts as an intermediary between the user and the
operating system. It interprets commands entered by the user and executes them. Here are some
key concepts related to the shell:
Types of Shell:
There are various types of shells available in UNIX-like operating systems, including:
• Bourne Shell (sh): The Bourne Shell was the original Unix shell developed by Stephen
Bourne. It provides basic functionality and is the foundation for many other shells. It has a
simple syntax and is commonly used for scripting.
• C Shell (csh): The C Shell was developed by Bill Joy as an improvement over the Bourne Shell.
It includes features inspired by the C programming language, such as command history and a
C-like syntax. It is known for its interactive capabilities and is popular among programmers.
• Korn Shell (ksh): The Korn Shell was developed by David Korn as an extension of the Bourne
Shell. It incorporates features from both the Bourne Shell and the C Shell, offering a
powerful and flexible shell with advanced scripting capabilities. It is widely used on Unix-like
systems.
• Bash (Bourne Again Shell): Bash is the default shell for most Unix-like systems, including
Linux. It is based on the Bourne Shell but includes additional features and improvements.
Bash is highly compatible with the Bourne Shell and provides interactive features, command-
line editing, and advanced scripting capabilities.
• Zsh (Z Shell): Zsh is an extended shell with powerful features and customization options. It
includes enhancements over Bash, such as improved tab completion, spelling correction,
and a flexible configuration system. Zsh is designed to be user-friendly and highly
customizable.
Pattern Matching:
The shell supports pattern matching, also known as wildcard expansion, where special characters
can be used to match multiple files or directories. Common wildcard characters include '', '?' and '[ ]'.
For example, '.txt' matches all files ending with '.txt'.
Escaping:
Escaping is the process of treating special characters as literal characters by preceding them with a
backslash (''). It allows you to use characters that would otherwise be interpreted as part of the
shell's syntax. For example, to use a literal '$' character, you would escape it as '$'.
Quoting:
Quoting is used to preserve the literal meaning of characters or strings. There are three types of
quoting in the shell: single quotes (' '), double quotes (" "), and backticks ( ). Single quotes preserve
the literal value of all characters, double quotes preserve most characters except for a few special
ones, and backticks are used for command substitution.
Redirection:
Redirection allows you to control where the input comes from and where the output goes. The most
common redirection operators are '>', '>>', and '<'. For example, 'command > file' redirects the
output of 'command' to 'file', overwriting its contents if it already exists.
'/dev/null' is a special file that discards any data written to it. It is commonly used to suppress
output. '/dev/tty' represents the current terminal device and can be used to read input or display
output from the terminal.
Pipe:
A pipe ('|') allows you to connect the output of one command to the input of another command. It
enables the chaining of commands, where the output of the preceding command serves as input for
the next command.
tee:
The 'tee' command reads from standard input and writes to both standard output and specified files.
It is often used in conjunction with pipes to display output on the terminal while simultaneously
saving it to a file.
Command Substitution:
Command substitution is a feature in Unix shells that allows you to use the output of a command as
part of another command or store it in a variable. There are two syntaxes for command substitution:
Using backticks ( ): You can enclose a command within backticks to substitute its output. For
example:
current_date=`date`
Using $( ): You can enclose a command within $( ) to achieve the same result. For example:
current_date=$(date)
In both cases, the command within the backticks or $( ) is executed, and its output is substituted in
place. This allows you to capture the output of a command and use it as an argument to another
command or assign it to a variable.
Shell Variables:
Shell variables are used to store data within the shell's environment. They can hold values such as
strings, numbers, or file paths, and are accessed using the variable name. Here are some key points
about shell variables:
Variable Assignment: To assign a value to a variable, use the syntax variable_name=value. For
example:
name="John"
age=25
Variable Expansion: To access the value stored in a variable, prefix the variable name with a dollar
sign ($). For example:
Built-in Variables: The shell provides various built-in variables that contain information about the
environment, such as $HOME (user's home directory), $PATH (search paths for executables), and
$PWD (current working directory).
Environment Variables: Environment variables are global variables that are accessible to all
processes in the shell session. They are typically set in shell configuration files like ~/.bashrc or
~/.bash_profile.
Local Variables: Local variables are specific to the current shell process or script and are not
accessible by other processes.
Read only Variables: You can declare variables as read only using the read only command to prevent
their values from being changed.
Exporting Variables: To make a variable available to child processes, you can use the export
command. For example:
Shell variables are a fundamental part of shell scripting and are used to store and manipulate data
within scripts. They provide flexibility and allow for dynamic behavior in shell programs.
Process
In UNIX, a process refers to an instance of a running program. It is a fundamental concept in
operating systems, including UNIX-like systems. Here is a basic idea about UNIX processes and
related concepts:
Basic Idea about UNIX Process:
A process represents the execution of a program or command on the system. Each process has its
own unique process ID (PID) assigned by the operating system. Processes can run in the background
or foreground. Processes can interact with the operating system, other processes, and system
resources.
The ps command is used to display information about running processes. It allows you to view
attributes such as the process ID, parent process ID, CPU and memory usage, and more. The ps
command has various options that allow you to customize the output based on your requirements.
To display system processes, you can use the ps command with specific options. For example, ps -ef
displays all processes running on the system, including system processes. The output provides
details such as the process ID, parent process ID, CPU usage, and command being executed.
In UNIX, the process creation cycle involves several system calls that are used to create, manage,
and terminate processes. Here is a brief explanation of the key system calls involved in the process
creation cycle:
• Fork: The fork system call is used to create a new process by duplicating an existing process.
It creates an identical copy of the parent process, including its code, data, and open file
descriptors. The new process is referred to as the child process, and it starts execution from
the point where the fork call was made. The fork call returns different values in the parent
and child processes: the child process receives a return value of 0, and the parent process
receives the process ID (PID) of the child process.
• Exec: After the fork call, the child process can use the exec family of system calls to replace
its memory space with a new program. The exec call loads the specified program into the
process's memory and starts executing it. There are several variations of the exec call, such
as execve, execl, execle, etc., each with different ways of specifying the program name and
command-line arguments.
• Wait: The wait system call is used by the parent process to wait for the termination of its
child process. It suspends the execution of the parent process until one of its child processes
exits. The wait call can also retrieve information about the child process, such as its exit
status, which can be used for error handling or further processing.
• Exec (again): If the child process wants to execute a different program after the initial exec
call, it can use another exec call to replace its memory space with the new program. This
allows for process chaining, where one program spawns another program for execution.
• Exit: The exit system call is used to terminate a process. When a process calls exit, it releases
its resources and notifies the operating system that it has completed its execution. The exit
status, which is an integer value, can be used by the parent process or other processes to
determine the outcome of the child process's execution.
Process State:
In UNIX-like systems, a process can be in different states that indicate its current status. The
common process states are:
Zombie State:
When a process completes its execution, it becomes a zombie process until its parent process
collects its exit status using the wait system call. The zombie process remains in the process table
with an entry but does not consume any system resources except for the process table entry. The
zombie process is essentially waiting for the parent process to acknowledge its termination and
collect the necessary information. Once the parent process collects the exit status, the zombie
process is removed from the process table.
In Unix-like operating systems, background jobs refer to processes that are executed independently
of the current terminal session. They do not require the user's direct interaction and continue
running even when the terminal is not actively used. There are a few ways to run jobs in the
background:
When executing a command, appending an ampersand (&) at the end of the command line instructs
the shell to run the command in the background. For example:
$ command &
The command will start executing in the background, allowing you to continue using the terminal for
other tasks.
The nohup command is used to run a command that continues running even after the user logs out
or the terminal session ends. It prevents the process from being terminated when the shell session is
closed. For example:
The nohup command redirects the standard output and standard error of the command to a file
named nohup.out, allowing the command to continue running in the background.
If a job is suspended or stopped, you can use the bg command to move it to the background and
resume its execution. The bg command is typically used with the job ID or job specifier. For example:
$ bg %job_id
The disown command is used to remove a job from the shell's job control. This allows the job to
continue running even if the shell session ends. For example:
$ disown %job_id
This command removes the specified job from the shell's control.
Background jobs are useful when you want a process to run independently without blocking the
terminal. They are commonly used for long-running tasks, batch jobs, or tasks that do not require
user interaction.
In Unix-like operating systems, the nice command is used to adjust the priority of a running process.
It allows you to modify the scheduling priority of a process, thereby affecting its CPU usage and
resource allocation. The nice command is typically used to lower the priority of a process, giving it
less CPU time and resources compared to other processes.
The nice command takes a value or a priority level as an argument, which ranges from -20 to 19. A
lower value represents a higher priority, and a higher value represents a lower priority. The default
priority value is 0.
In Unix-like operating systems, signals are used to communicate with processes and manage their
behavior. One common use of signals is to terminate or kill a process. The kill command is used to
send signals to processes, allowing you to control their execution. The most commonly used signal
for terminating a process is SIGTERM (signal number 15), which requests the process to terminate
gracefully.
To send a signal to a process using the kill command, you need to know the process ID (PID) of the
target process. You can obtain the PID using the ps command or other process listing utilities. The
basic syntax of the kill command is as follows:
[signal]: Specifies the signal to send. If not specified, the default signal is SIGTERM.
For example, to send the SIGTERM signal to terminate a process with PID 1234, you can use the
following command:
$ kill 1234
If a process does not respond to the SIGTERM signal and needs to be forcibly terminated, you can
send the SIGKILL signal (signal number 9), which immediately terminates the process without
allowing it to clean up.
Now, let's discuss sending a job to the background (bg) and foreground (fg):
When a process is running in the foreground, it occupies the terminal, and you cannot execute other
commands until it finishes or is suspended. To move a running job to the background, you can
suspend it by pressing Ctrl + Z. This will stop the job and return control to the shell. Then, you can
use the bg command to send the job to the background and allow it to continue its execution. The
basic syntax is as follows:
$ bg %job_id
%job_id: Specifies the job identifier of the suspended job. You can get the job ID from the jobs
command.
When a job is running in the background, you can bring it back to the foreground to make it the
active job. This allows you to interact with the job directly on the terminal. The basic syntax of the fg
command is as follows:
$ fg %job_id
%job_id: Specifies the job identifier of the background job. You can get the job ID from the jobs
command.
Once you bring a job to the foreground, it will resume execution, and you can interact with it as if it
were running in the foreground from the beginning.
These commands are useful for managing multiple jobs and controlling their execution in the
foreground or background, providing flexibility and control over the execution of processes in a
Unix-like environment.
In Unix-like operating systems, the jobs command is used to list the status of currently running jobs
or processes associated with the current shell session. It displays information about both foreground
and background jobs, including their job IDs and status.
$ jobs
When you execute the jobs command, it lists the jobs along with their job IDs (in square brackets)
and their current status. The status can be one of the following:
Stopped: The job has been suspended or stopped and is not currently executing.
To suspend a running job and move it to the background, you can press Ctrl + Z while the job is
running in the foreground. This sends the SIGTSTP signal to the process, temporarily stopping its
execution. The job will be listed as stopped when you run the jobs command.
To terminate a job, you can either use the kill command or press Ctrl + C while the job is running in
the foreground. Pressing Ctrl + C sends the SIGINT signal to the process, instructing it to terminate.
Alternatively, you can use the kill command followed by the job ID to send a specific signal to the
job.
$ kill %job_id
Here, %job_id is the job identifier obtained from the jobs command.
The at and batch commands in Unix-like systems allow you to schedule the execution of commands
or scripts at a specified time in the future.
at: The at command allows you to schedule a one-time job to be executed at a specific time. You can
provide the time using various formats, such as HH:MM, HH:MM AM/PM, or even a specific date.
You can then provide the command or script to be executed.
For example:
$ at 10:30 PM
at> command
at> Ctrl+D
batch: The batch command is similar to at, but it is designed for executing jobs when the system
load is low. The commands or scripts specified with batch are queued and executed when the
system load average falls below a certain threshold.
Both at and batch are useful for automating tasks and executing commands or scripts at specific
times or when the system load allows.
These commands provide flexibility in managing and controlling the execution of jobs, allowing you
to list, suspend, kill, and schedule jobs in a Unix-like environment.
Customization
Environment variables are variables that are part of the operating system's environment and are
accessible to all processes running on the system. They contain information that can be used by
programs or scripts to customize their behavior or access specific resources. Environment variables
are typically set and managed by the shell.
HOME: Specifies the home directory of the current user. It is typically set to the user's personal
directory, where their files and configurations are stored.
PATH: Defines the directories where the system looks for executable files. It is a colon-separated list
of directories, and when you enter a command in the shell, it searches for the executable in these
directories.
TERM: Specifies the terminal type or emulation being used. It is used by programs to adapt their
behavior to the capabilities of the terminal.
PWD: Stores the current working directory, which is the directory the user is currently in.
PS1 and PS2: Define the primary and secondary prompts, respectively, for the shell. They determine
the appearance and content of the command prompt.
In Unix-like systems, PS1 and PS2 are environment variables that define the primary and secondary
prompts, respectively, for the shell. These variables determine the appearance and content of the
command prompt, providing information and visual cues to the user.
PS1 is the environment variable that defines the primary prompt. It is displayed when the shell is
ready to accept a new command. The primary prompt typically includes information such as the
username, hostname, current working directory, and other customizable elements.
The primary prompt is usually set in the shell's configuration file (e.g., ~/.bashrc, ~/.bash_profile).
The default value for PS1 varies depending on the Unix-like system and the user's configuration.
PS1="\u@\h:\w\$ "
In this case, \u represents the username, \h represents the hostname, \w represents the current
working directory, and \$ represents the dollar sign as the regular user prompt.
PS2 is the environment variable that defines the secondary prompt. It is displayed when the shell
expects more input from the user because the command entered is incomplete or requires
additional input, such as an unclosed quotation or a multi-line command.
The secondary prompt is typically set to a simple indicator, such as > or :. This prompts the user to
continue typing the command or provide the required input.
The PS2 variable can also be set in the shell's configuration file, similar to PS1.
PS2="> "
By customizing the values of PS1 and PS2, users can personalize their command prompts to display
useful information or visual cues that suit their preferences or workflows.
In addition to environment variables, there are also aliases. Aliases are user-defined shortcuts for
commands or command sequences. They allow you to create a custom command or shorten lengthy
commands for easier and quicker execution. Aliases are defined using the alias command and are
typically stored in the shell's configuration files.
Command history is another useful feature provided by the shell. It allows you to access and reuse
previously executed commands. Each time you run a command, it is recorded in a history list, and
you can navigate through the history using keyboard shortcuts or commands. This helps you avoid
retyping frequently used commands and improves productivity.
The command history is usually stored in a file, such as ~/.bash_history, and the shell provides
various commands and shortcuts to interact with the history, such as history, !, and the up and down
arrow keys.
Overall, environment variables, aliases, and command history are essential features of the Unix shell
that allow users to customize their environment, define shortcuts, and easily access previously
executed commands.
Filters in Unix-like systems are commands that perform various operations on text files or streams.
They are used to modify, format, or extract specific information from files. Here are some commonly
used filters:
• pr: The pr command is used to prepare a file for printing by formatting it into pages with
headers, footers, line numbering, and other options.
• head and tail: The head command displays the first few lines of a file, while the tail
command displays the last few lines. These commands are useful for quickly inspecting the
beginning or end of a file.
• cut: The cut command is used to extract specific columns or fields from a file. It allows you
to specify the delimiter and the field numbers or ranges to extract.
• paste: The paste command is used to merge lines from multiple files side by side. It
concatenates corresponding lines from each file, separated by a specified delimiter.
• sort: The sort command is used to sort the lines of a file in either ascending or descending
order. It can sort based on various criteria, such as alphabetical order, numeric order, or
specific fields within each line.
• uniq: The uniq command is used to find and filter repeated or non-repeated lines in a sorted
file. It removes adjacent duplicate lines or displays only the unique lines, depending on the
options used.
These filters provide powerful capabilities for manipulating and analyzing text files in Unix-like
systems. By combining these filters with pipes (|) and redirecting input/output, you can perform
complex operations and create custom workflows to process and manipulate text data efficiently.
The tr command in Unix-like systems is used to manipulate characters in text files or streams. It
performs character translation or deletion based on the specified rules. Here are some common use
cases of tr:
Character Translation:
This command translates characters from the "source" set to the corresponding characters in the
"destination" set. It reads from the input.txt file and writes the translated output to the output.txt
file.
Character Deletion:
This command deletes the specified characters from the input. It reads from the input.txt file,
removes the specified characters, and writes the result to the output.txt file.
The grep command in Unix-like systems is used to search for patterns within text files or streams. It
scans input line by line and displays lines that match the specified pattern. Here's the basic usage of
grep:
For example, to search for the word "example" in a file called "text.txt":
By default, grep performs case-sensitive matching. If you want to perform case-insensitive matching,
you can use the -i option:
For example, to recursively search for the word "example" in all files within a directory and its
subdirectories, and display the filenames along with the matching lines:
The grep command is a powerful tool for searching and filtering text data based on patterns. It
supports regular expressions, allowing for more complex and flexible pattern matching.
Regular expressions (regex) are patterns used for matching and manipulating text. In Unix-like
systems, there are different flavors of regular expressions available, including Basic Regular
Expression (BRE), Extended Regular Expression (ERE), and a combination of grep -E or egrep for
extended pattern matching.
BRE is the default regex syntax used by many Unix utilities, including grep.
For example, to search for lines starting with "Hello" in a file using BRE:
ERE is an extended version of regular expressions that provides additional metacharacters and
features.
It is used by tools like egrep, awk, and sed when the -E option is specified with grep.
For example, to search for lines containing "apple" or "banana" in a file using ERE:
In summary, Basic Regular Expression (BRE) is the default regex syntax used by grep, while Extended
Regular Expression (ERE) provides additional metacharacters and features. egrep and grep -E are
used to enable extended regex matching. The choice between BRE and ERE depends on the specific
requirements of the pattern you are trying to match or search for.
4. Logical Operators:
a. Shell scripts support logical operators like && (AND) and || (OR) for conditional
execution of commands.
b. The && operator executes the next command only if the previous command
succeeds, while the || operator executes the next command only if the previous
command fails.
5. Condition Checking:
a. Shell scripts use conditionals like if and case to perform different actions based on
specific conditions.
b. The if statement allows you to perform actions based on the result of a condition.
c. The case statement provides a way to match a value against multiple patterns and
execute corresponding actions.
6. Expression Evaluation:
a. Shell scripts can evaluate expressions using the test command or the [ ] syntax.
b. The test command or [ ] can check conditions such as file existence, numeric
comparisons, string comparisons, and more.
7. Computation:
a. The expr command can be used for simple arithmetic computations within shell
scripts.
b. It supports addition, subtraction, multiplication, and division operations.
8. Loop:
a. Shell scripts use loops like while and for to iterate over a set of values or perform
actions repeatedly.
b. The while loop executes a block of code as long as a specified condition is true.
c. The for loop iterates over a list of values and executes a block of code for each value.
9. Positional Parameters:
a. Shell scripts can access the command line arguments using positional parameters
like $1, $2, etc.
b. These parameters allow you to pass values to the script when executing it.
Shell scripting provides a powerful way to automate tasks, create custom utilities, and perform
complex operations using a combination of shell commands, conditionals, loops, and variables. It is a
versatile tool for system administration, automation, and scripting tasks in Unix-like systems.
System Administration
The duties of a UNIX system administrator include managing and maintaining the UNIX-based
systems in an organization. Here are some essential duties and responsibilities of a UNIX system
administrator:
The user account management aspect of system administration involves creating and managing user
accounts on the UNIX system. This includes:
Overall, UNIX system administrators are responsible for the smooth operation, security, and
maintenance of UNIX systems, ensuring the availability and reliability of the systems for users and
applications.