Official CompTIA Linux+ Student Guide
Official CompTIA Linux+ Student Guide
CompTIA
Linux+
Student Guide
(Exam XK0-005)
Acknowledgments
Notices
Disclaimer
While CompTIA, Inc. takes care to ensure the accuracy and quality of these materials, we cannot guarantee their accuracy,
and all materials are provided without any warranty whatsoever, including, but not limited to, the implied warranties of
merchantability or fitness for a particular purpose. The use of screenshots, photographs of another entity’s products, or
another entity’s product name or service in this book is for editorial purposes only. No such use should be construed to imply
sponsorship or endorsement of the book by nor any affiliation of such entity with CompTIA. This courseware may contain links
to sites on the Internet that are owned and operated by third parties (the “External Sites”). CompTIA is not responsible for
the availability of, or the content located on or through, any External Site. Please contact CompTIA if you have any concerns
regarding such links or External Sites.
Trademark Notice
CompTIA®, Linux+®, and the CompTIA logo are registered trademarks of CompTIA, Inc., in the U.S. and other countries.
All other product and service names used may be common law or registered trademarks of their respective proprietors.
Copyright Notice
Copyright © 2022 CompTIA, Inc. All rights reserved. Screenshots used for illustrative purposes are the property of the software
proprietor. Except as permitted under the Copyright Act of 1976, no part of this publication may be reproduced or distributed
in any form or by any means, or stored in a database or retrieval system, without the prior written permission of CompTIA,
3500 Lacey Road, Suite 100, Downers Grove, IL 60515-5439.
This book conveys no rights in the software or other products about which it was written; all use or licensing of such software
or other products is the responsibility of the user according to terms and conditions of the owner. If you believe that this
book, related materials, or any other CompTIA materials are being reproduced or transmitted without permission, please call
1-866-835-8020 or visit https://help.comptia.org.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Table of Contents
Table of Contents
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Table of Contents
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Solutions......................................................................................................................... S-1
Glossary...........................................................................................................................G-1
Index................................................................................................................................. I-1
Table of Contents
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Course Description
Course Objectives
This course can benefit you in two ways. If you intend to pass the CompTIA Linux+
(Exam XK0-005) certification examination, this course can be a significant part of
your preparation. But certification is not the only key to professional success in
the field of systems administration. Today’s job market demands individuals with
demonstrable skills, and the information and activities in this course can help you
build your sysadmin skill set so that you can confidently perform your duties in any
intermediate-level Linux systems administration role.
On course completion, you will be able to:
• Configure, manage, and troubleshoot Linux systems.
Target Student
The Official CompTIA Linux+ (Exam XK0-005) is the primary course you will need to
take if your job responsibilities include Linux system administration, installation,
and security within your organization. You can take this course to prepare for the
CompTIA Linux+ (Exam XK0-005) certification examination.
Prerequisites
To ensure your success in this course, you should have at least 12 months of
hands-on experience working with Linux servers. CompTIA A+, Network+, and
Server+ certifications, or the equivalent knowledge, are strongly recommended.
The prerequisites for this course might differ significantly from the prerequisites for
the CompTIA certification exams. For the most up-to-date information about the exam
prerequisites, complete the form on this page: www.comptia.org/training/resources/
exam-objectives.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
As You Learn
At the top level, this course is divided into Lessons, with each representing an area
of competency within the target job roles. Each Lesson is composed of a number of
topics. A Topic contains subjects that are related to a discrete job task and mapped
to objectives and content examples in the CompTIA exam objectives document.
Rather than follow the exam domains and objectives sequence, lessons and topics
are arranged in order of increasing proficiency. Each topic is intended to be studied
within a short period (typically 30 minutes at most). Each topic is concluded by one
or more activities, designed to help you apply your understanding of the study
notes to practical scenarios and tasks.
In addition to the study content in the lessons, there is a glossary of the terms and
concepts used throughout the course. There is also an index to assist in locating
particular terminology, concepts, technologies, and tasks within the Lesson and
topic content.
In many electronic versions of the book, you can click links on key words in the topic
content to move to the associated glossary definition and on page references in the
index to move to that term in the content. To return to the previous location in the
document after clicking a link, use the appropriate functionality in your eBook viewing
software.
As You Review
Any method of instruction is only as effective as the time and effort you, the
student, are willing to invest in it. In addition, some of the information that you
learn in class may not be important to you immediately, but it may become
important later. For this reason, we encourage you to spend some time reviewing
the content of the course after your time in the classroom.
Following the lesson content, you will find a table mapping the lessons and topics to
the exam domains, objectives, and content examples. You can use this as a checklist
as you prepare to take the exam and review any content that you are uncertain
about.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
As a Reference
The organization and layout of this book make it an easy-to-use resource for future
reference. Guidelines can be used during class and as after-class references when
you’re back on the job and need to refresh your understanding. When taking
advantage of the glossary, index, and table of contents, you can use this book as a
first source of definitions, background information, and summaries.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
Working with Linux begins with an understanding of licensing and the operating
system’s history. The open-source nature of Linux has resulted in many different
distributions, so it’s important to understand how distributions differ from each
other. Linux servers are primarily managed from the command line, using shells
such as Bash. Bash enforces a particular syntax, or way of structuring commands.
In addition, Linux holds its configurations in text files, so it’s critical that sysadmins
can edit these files to manage system settings. Man pages are available as
quick reference documents to help administrators recall the function of specific
commands and any available options.
Misconfigurations or physical failures may provide troubleshooting opportunities,
so sysadmins should follow a standard methodology to help narrow the scope of
problems, solve the root cause of the issue, and manage documentation related to
configuration issues.
Lesson Objectives
In this Lesson, you will:
• Identify Linux characteristics.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 1A
Identify Linux Characteristics
2
Some examples of open-source licenses include the Apache License 2.0, the GNU
General Public License, and Mozilla's Public License. While the exam does not focus on
the specifics of these licenses, they are a good sample of the open-source requirements
and permissions.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Of the three primary operating systems in the marketplace today (Linux, macOS,
and Windows), two can trace their roots back to Unix. The macOS kernel evolved
from a Unix flavor named BSD and shares many of the same standards and some
software as Linux. However, Apple’s OS is not FOSS. Microsoft Windows also uses a
proprietary kernel with a more restrictive licensing method.
The timeline of early OS development. Unix, released in 1969, directly generated the Linux and the
macOS systems.
• Learning curve: Some find that Linux has a steeper learning curve than
Windows or macOS does.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Software management, including more differences between the Red Hat method and the
Debian method, are discussed in a later section.
• Linux Mint
• Debian
• openSUSE
After the release of Linux in 1991, the two major branches, Debian and Red Hat, followed quickly
and generated hundreds of distros.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• Name resolution: Hosts Domain Name System (DNS) name resolution services.
Most commands are consistent across distributions. A few commands, such as those
for software management, may be specific to one group of distributions or another. For
example, Red Hat Linux uses the rpm command to manage software, while Debian
Linux uses apt.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• Scriptable: CLI commands can be written into a text file, which the system then
reads and executes in a consistent, efficient, repeatable, and scheduled manner.
CLI disadvantages:
• Learning curve: Remembering many different commands and their related
options is difficult.
• Inconsistent: Many commands differ from each other in small but distinctive
ways, making it difficult to recall exactly how to use them.
Common CLIs
Command-line interfaces are available in Linux, Windows, and macOS. Users type
commands using a specific syntax, and the system processes the commands. At
first, such input may seem intimidating or difficult, but CLI environments get easier
with use. These environments are usually faster and offer automation options that
are not available in GUIs.
Several sample commands and their output, including whoami, pwd, and date.
Shells provide the CLI. Each shell has its own syntax, or way of structuring
commands.
Common Linux shells:
• Bash: Default Linux shell
• zsh: Z shell
The Bash shell is covered in more detail later in this Lesson. It is the only shell covered
by the CompTIA Linux+ exam objectives.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Common GUIs
Just as there are many different Linux distributions, there are also many different
Linux graphical environments. Windows and macOS users have one GUI available
to them—whatever graphical environment Microsoft and Apple choose to provide.
Linux users have the freedom to install zero, one, or many GUI environments and
switch between them.
These GUIs are usually distinguished by two characteristics: user-friendly interface
and performance. Some users like the look and feel of a particular GUI over others.
In addition, some GUIs consume more processor time and memory than others do.
Luckily, many options are available in the Linux world.
Common GUI environments include GNOME, KDE Plasma, Cinnamon, and MATE.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
3
Linux Characteristics
Answer the following questions:
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 1B
Understand Bash Interaction
with Linux
5
Command Shells
The CLI is provided by software called a shell. The shell accepts user input,
processes the input for syntax, and provides output back to the user. The default
shell for most Linux distributions is Bash, and this is the shell that sysadmins
should be prepared to work with.
Other common Linux shells include ksh, or KornShell, which is common among
Unix servers; Zsh, or Z Shell, with quite powerful scripting capabilities; and Fish, or
friendly interactive shell, an interface that provides a user-friendly experience and
web-based configurations.
By way of comparison, Windows Server also uses shells: the traditional, DOS-like
cmd.exe shell and Microsoft PowerShell. The current (at the time of this writing)
default shell for macOS is the Zsh.
Bash is the Linux default and the only shell to concern yourself with for CompTIA Linux+.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• Option: A command modifier that slightly changes the way a command is processed.
• Argument: The object on which the command acts. For example, in a command
to delete a file, the argument is the name of the file to be deleted.
There are two basic forms, normal command and command-subcommand, to this
syntax.
As an example, here are several ways to use the list (ls) command with options
and arguments.
Most Bash error messages are descriptive, so be careful to read the error message to
understand what went wrong.
Command-Subcommand Syntax
Many Linux commands support subcommands to specify particular information
that the sysadmin needs. These commands rely on a different syntax from the basic
format in normal command syntax. The sysadmin enters the primary command,
then follows it with a space and a subcommand, and then a space and argument.
The ip command uses this format.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Example with
Command Purpose Options Result
ls List the contents of the ls /tmp List the contents of
current directory the /tmp directory
touch Create a new empty file touch Create a new file
or update the timestamp newfile.txt named newfile.txt
on an existing file
cd Change from one cd /etc Changes the
directory to another current directory
to /etc
cat Display the contents of a cat data.txt Display the
text file on the screen contents of the
data.txt file
less Display the contents of a less Display the
file in windows that fit on data.txt contents of the
the screen data.txt file screen
at a time when
the file would not
normally fit on one
screen
tree Display the directory tree /etc Display the
structure in a tree subdirectories and
format files in the /etc
directory in a tree
structure
shutdown Shut down the system shutdown Restart the system
-r now immediately
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Two common commands do not use options to generate an output. Use whoami
to display the current user, and use pwd to display the present working directory.
Command line interface showing the output of ls, pwd, whoami, and touch.
The number of Bash commands can be overwhelming. Start by using a few commands
at a time, and make them a habit. The longer you work with Linux, the more
comfortable you’ll become with the commands.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Typing the history command displays the contents of the history file. Each entry in
the file is numbered. Type ! and the command number executes that command.
Some shells cache command history in memory. When the system is rebooted, the
commands are no longer available. Bash writes the command history to a file stored on
the hard disk. The commands are available even after multiple reboots.
• Use command history instead of rewriting long commands: When you make
a typographical error in a command or file name, do not manually retype the
entire line. Repeat the line with the mistake by hitting the Up Arrow key one
time, and then use the Left and Right Arrow keys to move to the mistake so that
you can correct it.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• Clear the screen: Enter the clear command to clear the CLI of all text. This
is useful when you’re starting a new task and want to eliminate any distracting
information from past command entries.
Vim
Vim is very powerful and complex. It uses three different modes, where each
mode maps keyboard keys to different functions. For example, in Insert mode
the keyboard acts as normal, inserting text into the file. If you’re in Insert mode
and type “abc,” those three characters appear in the file’s content. In Command
mode, pressing a key on the keyboard issues commands to Vim instead of entering
text in the file. Selecting the i key tells Vim to switch from Command mode to
Insert mode. The third mode is Execute. This mode is entered by selecting the
colon character, : , and it provides a command prompt to Vim where additional
commands can be issued. For example, :wq places Vim in Execute mode, writes
the files to the disk (save), and then quits Vim (q).
The many modes and commands can make Vim a little confusing. Strive to
understand four basic functions: create/open, edit, save, close.
For efficiency, type :wq to combine the save and close options.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The Vim editor. Note the INSERT flag in the lower left corner.
Nano
Nano is a popular and common alternative to Vim. It’s simpler but less powerful.
However, in many cases sysadmins don't need the power offered by Vim, which
makes Nano a useful choice for basic editing functions.
Nano does not have modes. Pressing keys on the keyboard inserts text into the
file, just as expected with most editors. To save and close the file, use keyboard
shortcuts using the Ctrl meta key. For example, Ctrl+O saves the file, and
Ctrl+X exits the file. You may have used similar keyboard shortcuts in other
applications.
As with Vim, it’s critical that you are capable of using Nano to create or open, edit,
save, and close files.
• Type nano filename to create a new empty file or open an existing file with
Nano.
• To edit the file, simply begin typing. Use the arrow keys to move the cursor.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Some Linux distributions install both Vim and Nano by default, while others will
include only one or the other. It is essential for you to be able to use both editors at
a very basic level (open, edit, save, close) so that you are capable of editing files with
whichever tool is available.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Elevate privileges from standard user to root, and confirm the change with
the whoami command.
The problem with the su - root command is that it grants all administrative
privileges to the escalating user, assuming the user knows the root password.
In other words, the user is either a non-privileged account with almost no
administrative authority or the full root user account with all possible administrative
authoring—and nothing in between. Sometimes, administrators want to delegate
specific, defined activities that require root authority, but only those particular
activities.
Sysadmins can edit a file named /etc/sudoers to delegate specific tasks to
individual users and groups. The specified identity may exercise that task as if
they are root, but nothing else. This is a much safer alternative than giving full
root privileges to individuals who may not be fully qualified to run the system. This
delegation concept is critical to good security.
To accomplish a delegated task, simply precede the command with sudo. You will
usually be prompted for your password and given a warning to be careful on the
system. The command then executes.
As a security measure, some distributions disable the root user account and force
the use of sudo on specific user accounts.
Privilege escalation using su and sudo are covered in more detail in a later Lesson.
A summary is provided here in case it’s needed for hands-on activities.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• /home/username: Each standard user has a specific and private directory used
to store personal files, profile settings, and other data. These user directories are
subdirectories of /home.
• /etc: Most system configuration files are stored in the /etc directory.
• /var/log: Log files for the system and applications are stored in the /var/log directory.
There are many other standard directories, and they are covered in a later Lesson.
If the system boots to the GUI, a login prompt is displayed that may show available user
accounts. A password is entered, and then the user is authenticated. Profile settings
related to the GUI—such as desktop backgrounds and menu items—are then loaded.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Interact with Linux
6
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 1C
Use Help in Linux
4
Linux Documentation
There are several ways of getting help in Linux. The most common are the manual
pages, referred to as “man pages” for short. There is built-in documentation for the
system and some applications, too. Many online resources also exist, and they are
often the most up to date.
Because there are so many commands, and because each command has so many
options, it’s very common to use the man pages as a quick reference for displaying
the available options.
Manual Pages
It’s common for new Linux users to ask for help and then be asked, “Did you check
the man pages?” That’s because man pages are the primary reference for standard
Linux commands. The man pages provide syntax information and usage examples.
Perhaps most important, the available options are displayed. Because of the
number of options for each command, and the fact that many options differ from
command to command, the man pages provide an essential quick reference.
The syntax for using man pages is man {command}.
For example, to display help for the ls command, type man ls.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Note that the numbers in this list are man page section numbers, not merely
part of the list. In other words, “System calls” is in Section 2 of the man pages
documentation, not just the second item in this grouping.
Section eight is probably the most commonly used section for administrators.
Built-In Documentation
Most commands include help references. Add the -h option, or help after the
command to display this reference material.
The whatis command provides a brief description of the specified command.
The syntax for whatis is whatis {command}.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Online Documentation
There is a great deal of information available online that covers Linux
administration, applications, security configurations, and network services. This
documentation may be provided by vendors, community groups, online forums,
article repositories, and other sites.
• Linux distribution vendors: Vendors such as Red Hat and Ubuntu have large
repositories of reference information.
• Linux application vendors: Vendors for products such as Apache web server,
Vim, and Firefox provide many references for their applications.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Help in Linux
5
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 1D
Identify the Linux Troubleshooting
Methodology
3
Troubleshooting Methodology
A formalized and consistent troubleshooting methodology can make identifying
issues and discovering fixes more efficient. While the steps can vary depending
on the actual issue and components involved, there are several universal
troubleshooting steps.
The following list represents the basic steps in a troubleshooting methodology:
• Identify the problem.
Throughout the process you will find it helpful to document findings, actions, and
outcomes of the various steps.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
In Linux, the log file service is named "rsyslog." Services are covered in Lesson 9.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
down to replace hardware, or if data has been lost due to a HDD failure, the end
users must be notified. The plan of action defines the steps to be taken. These steps
should be defined ahead of time rather than created during the implementation of
the solution. It is useful to provide the impacted users with an expected duration of
the outage.
Some service desk management software requires the use of tickets. Such software may
require that troubleshooting documentation be entered before the ticket can be closed.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Troubleshoot in Linux
4
1. A user contacts you to find out why they cannot access a directory. Using
the troubleshooting methodology, how would you narrow the scope of
the problem?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Lesson 1
Summary
4
Guidelines
These best practices and guidelines are provided for your use as revision tools or as
quick references in your job role.
• FOSS: Recognize and describe free and open-source software, including
advantages and disadvantages.
• GPL: Understand how the GPL influences the development and availability of the
Linux OS.
• Use Vim and Nano to open, edit, save, and close files.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
Modern operating systems require users to prove their identity. This identification
process, referred to as authentication, provides access to major functions. First, the
user’s identity can be used to control access to resources via permissions and other
rules. Second, the user’s environment can be customized to fit that individual’s work
style and habits.
The first task is to understand user account management, including its related
configuration files and commands. However, managing individual users can be
inefficient, so users are often placed into groups for simplicity. Group management
involves a different set of configuration files and commands. Next, administrators
should log on to the server with non-privileged accounts and then elevate privileges
only when necessary. The administration of privilege elevation and its related
commands is covered in the third Topic. Finally, this lesson will cover applying
configuration files and commands in troubleshooting and security auditing.
Lesson Objectives
In this lesson, you will:
• Manage user accounts.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 2A
Manage User Accounts
2
Sysadmins are responsible for, among other things, managing user accounts. These
accounts customize the user environment and provide access to resources. It’s
difficult to administer user accounts effectively without a solid understanding of the
files and tools used to manage accounts. This Topic covers user configuration files
and management commands.
Field Content
User Name The name the user logs into the system with
Password User password represented as an x; the actual password is
stored elsewhere
User ID Unique number representing the user to the system
Group ID Unique number representing the user’s primary group
Comment Typically displays the user’s full name
Home directory Absolute path to the user’s home directory
Login shell Absolute path to the user’s default shell (usually /bin/bash)
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The /etc/shadow file. Note the long string representing the hashed password.
Field Content
User name The name the user logs into the system
with
Password Hash value of the user’s password
Days since last password change Number of days since the last password
change; counted from January 1, 1970
Days before password may be Minimum changeable period, typically
changed set at 1 day
Days before password must be Maximum number of days since the last
changed password change before the password
must be changed again; a value of
99999 means the password never
needs to be changed, but often set at
90 days
Days until the user is warned to Days before the date the password
change password must be changed that the warning is
issued, often set to 7 days
Days after password expires that Number of days after the password
the account is disabled expires until the account is disabled;
should be immediate
Days until account expires Number of days until the account
expires and cannot be used
Unused field Reserved for potential future use
It may seem odd that account information is distributed across two files and that
specifically password information is stored in a different file than user accounts.
Passwords were originally stored in the second field of the /etc/passwd file.
This file, however, is “world-readable,” meaning that all users have read permissions
to the file. The password itself, or the encrypted version of it, could be viewed by all
users. To increase security, the password was moved to the /etc/shadow file,
which only the root user (administrator) can access.
The default shell setting, defined in the last field of the /etc/passwd file,
specifies which shell will launch when the user logs in. There are many different
shells, and some users may prefer one over another. Bash is the default shell and
therefore the most common.
For example, User1 has experience with several Unix flavors and is already familiar
with the Korn shell (ksh). The sysadmin may install ksh and set it as the default shell
for this user. The last field of the User1 line in /etc/passwd will read /bin/
ksh. User2, however, might be more comfortable with the Bash shell. In that case,
the sysadmin leaves the default shell value as /bin/bash.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The tilde character (~) represents the current user's home directory. For example, the
~ character means the same as /home/user1 (assuming user1 is the currently
logged-in user).
The system-wide files process first, and then the user-specific files are executed.
The user-specific configuration files take precedence over system files, allowing
users to customize their environments to suit their needs.
Finally, a sysadmin can store files in the /etc/skel directory and have those files
copy automatically to the home directory of any new user. This feature is useful for
pre-populating configuration files such as .bashrc with suggested settings for users
or for distributing policy documents all users should have access to.
The user management lifecycle, including adding, modifying, and deleting a user.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The default values found in the /etc/login.defs file can be overridden with
various options, providing administrators with flexibility. Common options include
the comment field or default shell settings.
Some common options for the useradd command include:
Observe that the comment value is enclosed in double quotes. The quotes cause Bash to
recognize the enclosed information as a single object. If the quotes did not exist, the first
name would be seen as a separate item from the last name, resulting in an error.
Set a Password
The useradd command creates the user but does not set a password. Most
Linux systems will not allow a user to log in with a blank password, so while the
account exists, it is not yet usable. The passwd command sets passwords for user
accounts.
The syntax for using passwd is passwd [username]
Helpdesk Ticket #01982
Submitted by: Department: Assigned to: Date Opened:
Kai Garcia Engineering you 2024-03-08
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
You are prompted to enter a new password twice. Use the passwd command to
configure a password for a new account as well as to reset a forgotten password for
an existing user.
The adduser command walks the admin through each field of the user account information.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Modify these existing user accounts by using the usermod command. In these
two cases, the commands will look like this:
# usermod -e 2025-12-31 alee
# usermod -c "Joseph Deng" jdeng
Helpdesk Ticket #01984
Submitted by: Department: Assigned to: Date Opened:
Kai Garcia Engineering you 2024-04-01
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The userdel command removes existing users from the system. By default, the
command does not remove the user’s home directory. This is important, as the user
data may need to be assigned to other users. However, the -r option can be added
to the command to remove the account and its associated home directory.
The syntax for using userdel is userdel {user-name}
For example, to delete the Alex Lee account, type userdel alee
Neither the usermod nor userdel commands will modify users if the accounts
have running processes.
The deluser command removes user accounts from the system on some
distributions.
Deleted user accounts cannot be recovered. They can be recreated, but they will be a
different identity (unless created with the same UID value). It is often better to disable a
user account rather than delete it.
Verification
When you use the useradd, usermod, or userdel commands to manage
users, the result of the command is recorded, even if nothing is displayed on
the screen. For example, when you create a user, there is no message displayed.
When something goes wrong, an error is displayed explaining the issue. While the
messages are self-explanatory, such as “Username already in use” informing you
that the username you tried to set exists already, the results are actually labeled
using an exit code number. Display the results of the most recent command by
typing the following:
# echo $?
0
A zero indicates success; any other value indicates an error of some sort.
Some examples of those exit codes are provided here.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The error values differ slightly for the usermod and userdel commands.
All Executables have exit codes. Use the echo $? command to display the exit status
of the most recent command.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
User Account Management
3
1. Why are user passwords stored in the /etc/shadow file and not the
/etc/passwd file?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 2B
Manage Group Accounts
4
Like the management of user accounts in the previous section, managing groups
requires an understanding of the configuration files and commands used to govern
groups. Groups simplify the process of granting access to resources to multiple
users.
Output of the tail /etc/group command, showing the most recently added groups.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
As the help desk tech, you run the following two commands to complete the service
request:
# groupmod -n publicity marketing
# groupadd sales
The -n option changes the group name.
Deleting a group does not delete the user accounts that are a member of that
group. A group is a separate object from the user.
Note that you cannot remove the primary group of an existing user. You must
remove the user account first by using the userdel command.
Some distributions rely on addgroup and delgroup to manage groups.
These commands are more interactive than groupadd and groupdel. If the
commands are not installed on the system, they can be added.
Be sure to search for files or other resources that may be associated with the group
before deleting it. Configure another group with access to those resources before
deleting the original group.
Verification
Group management commands are usually straightforward, but things can go
wrong. The messages printed on the screen should be plenty to guide you toward a
solution. However, you can also display the exit values by using echo $?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Group Account Management
5
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 2C
Configure Privilege Escalation
5
One should avoid logging on to a system as the root user. The root user’s broad
privileges are unnecessary for most day-to-day user tasks, such as managing data
or browsing the Internet. If the standard practice is to log on with a non-privileged
standard account, then what about situations where administrative privileges are
required? Instead of logging off and then logging back on as root, Linux provides
ways of switching identities or running commands as a different user.
This section introduces privilege escalation and then covers tools such as su,
sudo, and pkexec. Proper use of privilege escalation helps to maintain the
security of the system and satisfy security policy requirements.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Those distributions that do not disable the root user still suggest that sysadmins log
in with non-privileged accounts and then elevate privileges by using su.
Avoiding signing in as the root user is an example of the principle of least privilege. This
principle states that the minimum level of access should be granted to accomplish a
given task.
Note that in this discussion, the string "su - {user-name}" has a space on each side of the
dash. This is a rare instance in Linux where this is true; the majority of other commands
are spaced as "command -option," which has a space only on the left side of the dash.
Using su with a hyphen following it enables a user to change users and launch a
new shell under the context of that new user. This is a much better practice. Anyone
using the su - except the root user will be challenged for the password of the
user they are switching to. It is most common to switch to the root user, but any
user can switch to any other user as long as they know that user’s password.
Without a user name argument, the su - command will assume you mean to sign in
as root.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
You can check your sudo privileges by using the -l option. For example, type
sudo -l, and your privileges are displayed.
It may be difficult to get into the habit of using sudo before each command. Recall
that Bash keeps a history of recent commands and that !! repeats the most recent
command. If you forget to type sudo in front of a command, rerun the command by
typing sudo !!.
Understand visudo
While the /etc/sudoers file is a normal text file, it is essential not to directly
edit it with a standard text editor like Vim or nano. The /etc/sudoers file
controls access to all elevated privileges, and a mistake in this file can render it
impossible to gain root privileges on the server. Most distributions will set a default
editor (usually Vim or nano) for /etc/sudoers. When using the visudo
command, the system verifies the syntax of the /etc/sudoers file before
committing changes, enabling the administrator an opportunity to correct mistakes
before they become part of the running configuration.
The following are some options you can use with the visudo command:
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
PolicyKit Rules
The polkit (PolicyKit) toolkit provides a different way of delegating privileges than
sudo does. The polkitd daemon works with systemd to permit administrators to
delegate more granular control to users than sudo allows. Unlike sudo, polkit
does not grant full root access to a process but instead grants specific access to
defined actions.
One example of polkit’s benefits is software management. Using polkit rules,
sysadmins can permit a user to update existing software but prevent the
installation of new software. Other delegated tasks include system shutdown or
hibernation, configuring network devices, controlling device access, and mounting
or unmounting filesystems on removable media.
Some topics in this section, such as systemd and software management, are covered in
more detail elsewhere.
Use of pkexec to create a user account with confirmation that the account has been created.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
For example, to launch nano with pkexec and edit the /etc/named.conf
configuration file, type:
pkexec nano /etc/named.conf.
Recall that the su - root command actually changes the user identity to root,
with all related privileges. This could be dangerous. The sudo and polkit utilities
allow administrators to delegate specific tasks to specific users without granting
complete administrative privileges to the system.
Phone Call 1: su
A user calls and says that the su command is not responding as expected. You ask
the user to type whoami and the user reports the result is root. You also ask the
user to type pwd and the user reports the result is their own home directory. When
the user switches identities, they remain in their own home directory with their own
profile settings. The user wants to know what is preventing them from switching to
root and gaining the root user’s profile.
In this case, it’s likely that they are using the incorrect syntax. The user failed to
include the space on both sides of the dash between the su command and the
username, su - root.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Privilege Escalation
6
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 2D
Troubleshoot User and Group Issues
5
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Users may fail to authenticate to the system. These failures are often associated
with user account and password misconfigurations. Recall that account information
is stored in the /etc/passwd and /etc/shadow files.
2. The user enters a name and password combination. These are checked
against the /etc/passwd and /etc/shadow files. Settings such as
expired passwords and locked accounts are checked for at this point.
3. System and user profile files are processed, and the user is presented with an
authenticated and customized environment.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The last command displays the most recent logins and logouts.
It is a good security practice to delete unused user accounts, and these tools help
identify such accounts.
While the historical data displayed by last and lastlog can be helpful, it’s
also useful to know what users are currently on the system. Both the w and who
commands display this information. The w command is particularly useful because
it shows the user idle time, which indicates the user may be logged on but no longer
interacting with the session.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
User and Group Troubleshooting
6
1. List at least three scenarios where you might need records of who logged
in to a Linux system.
2. Another administrator asks you to explain the value of editing the /etc/
sudoer’s file with visudo rather than a traditional text editor. What is
your response?
5. A user places sudo before a command, but the command still fails to run.
What might be the cause?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Lesson 2
Summary
7
Observe how user authentication allows the system to uniquely identify the logged-
on user. The primary purpose of this identification is to enforce permissions on files
and directories. Permissions can be granted (or denied) to identities, allowing tasks
such as read-only file access, script execution, or application execution. Users are
placed into groups when more than one user must be restricted by permissions.
The use of user identities is further exemplified by considering privilege escalation.
What privilege escalation actually provides is the ability to move from one user
identity to another to exercise different levels of resource access.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
The identity of an authenticated user provides context for enforcing permissions.
Once the system knows which user is logged on, it can limit the actions that user
can take. Linux relies on a simple yet effective permissions structure that recognizes
three identities (users, groups, and all others) and provides three access levels
(read, write, and execute). More complex permissions options exist, including
special permissions and access control lists (ACLs).
Administrators manage group membership, file ownership, and group association
to control access to resources. As such, the permissions discussion in this Lesson is
a natural extension of the account management subject of the previous lesson.
Lesson Objectives
In this lesson, you will:
• Configure standard Linux permissions.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 3A
Configure Standard Linux Permissions
2
The authentication process provides users with an identity on the system. Based on
this identity, the system can enforce levels of access to system resources, such as
processes and files. This type of access control allows users to share systems and
resources with the appropriate levels of confidentiality and privacy.
Sysadmins rely on default Linux settings, such as umask, to define some levels of
access. Administrators also define explicit controls by associating specific access
levels with particular identities. Linux has several permissions structures, including
standard, special, and access control list mechanisms. Users are organized into
groups to make access control more efficient. Proper permissions settings and
owner/group associations are critical to Linux security and administration.
• Improper: They are granted the read, write, and execute permissions and could
potentially change the file.
Joseph Deng needs to create files associated with their own job but doesn’t need to
perform system administrative tasks.
• Proper: Joseph logs on with a standard user account.
Keep the principle of least privilege in mind in all access control configurations,
especially those involving file permissions.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Access Levels
Standard Linux permissions define three levels of access: read, write, and execute
(rwx). These permissions perform differently depending on whether they are
applied to files or directories.
Files Directories
Read (r) The ability to access and The ability to list the
view the contents of a file. contents of a directory.
Write (w) The ability to save changes The ability to create,
to a file. rename, and delete files
in a directory. Requires
the execute attribute to
also be set.
Execute (x) The ability to run a script, The ability to access a
program, or other software directory, execute a file
file. from that directory, or
perform a task on that
directory (e.g., a search).
The three permissions levels are abbreviated with r, w, and x. These abbreviations
are used in conjunction with permissions management commands, so be able to
recognize them.
Access Identities
Standard Linux permissions define three identities to which permissions can be
applied: user (owner), group, and others.
The user (u) identity is a single resource owner. Only one user identity is associated
with the resource with standard permissions, and specific permissions are applied
to this identity. By default, the user is the identity of the account that created the
resource, but that can be changed. This user identity is referred to as the resource
“owner.”
The chown command allows administrators to change the associated user (and group).
The chown command is covered later in this Topic.
The group (g) identity is a single group of users associated with the resource. In
standard Linux permissions, only one group is assigned to the resource and a
specific set of permissions is assigned. By default, the creator’s group is set on the
resource, but that can be changed.
A group allows multiple users to access the resource at a given level. For example,
a sales group consisting of several user accounts can be granted read access to a
sales file, allowing all members of the group to view the contents of the file.
The others (o) identity represents all accounts that are not the user (u) identity or
a member of the one associated group (g). The others identity means “everyone
else” who is not the user and not a member of the group. These other accounts
are granted a level of access. Frequently, that permissions level is either read or
no access.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Linux permissions are enhanced with special permissions and access control lists (ACLs),
which are both covered later in this Topic.
View Permissions
The ls command, introduced earlier, relies on the -l option to display
permissions. Think of the ls -l command as “list in long format,” where the
long format includes the display of permissions. Sysadmins use this command
frequently to confirm permissions settings.
The ls -l command displays details about permissions, owner, group, size, and more.
In the following example, note that the resource is a file (based on the leftmost
dash character), the owner (root) has rwx, the group (grp) has rwx, and all others
have r only.
-rwxrwxr-- root grp fileA
Interpreting Permissions Strings
The permissions string consists of 10 fields, with each field containing different
information.
The first field defines whether the resource is a file or a directory. A file is identified
by a dash character (-), while a directory is identified by a d character.
The first column of the permission string, listing one directory and three files.
The next three fields are related and identify the permissions assigned to the
resource for the user identity. If a permission is granted, it is identified with one of
three characters: r, w, or x. If the permission is not granted, the dash character is
used as a placeholder.
For example, if the user is granted read but not write or execute, the three fields
appear like this: r--. If the user is granted read and write, but not execute, the three
fields appear as rw-. Note that the permissions are always displayed in the rwx
order, and the - is a placeholder for any unassigned permission.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The second three columns list the permissions assigned to the user identity. Here the user
has rwx access to the directory and the first file, rw access to the second file, and read
only access to the third file.
The same structure applies to permissions assigned to the group, in the fifth, sixth,
and seventh fields.
The group permissions are outlined here. The group has r-x access to the directory, rw access to
the first and second files, and no access at all to the third file.
And the final three fields use the same structure for others.
The final three columns show permissions for others not in the user or group identity.
Permissions are more stringent here, with others having r-x access to the directory
and read only access to the first file, with no access at all to the last two files.
Additional fields, such as owner, group, and file size, have been removed from these
examples to provide clarity.
A file named file A, with the user having read, the group having read, and others
having no access would appear as:
-r--r----- fileA
A directory named dir1, with the user having read, write, and execute; the group
having read and execute; and others having read and execute would appear as:
drwxr-xr-x dir1
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
A directory named dir1, with the user having read and execute, the group having
read and execute, and others having no access would appear as:
dr-xr-x--- dir1
Recall that the execute (x) permission is required for the user to use cd to move into
the directory. If a user does not have execute to a directory, they cannot change into
that directory.
Default Permissions
The ls -l command displays the existing permissions, but what if permissions
need to be changed? Administrators can set file and directory permissions for all
resources, and resource owners (the account identified by the user (u) permissions)
can also configure permissions.
A value named umask defines default permissions on a per-user basis. The “change
mode” or chmod command changes permissions settings from the default values.
• 2 means that the group permissions should be masked by 2 (i.e., subtract 2 from
the default [6] and you get 4). Group members now only have read access.
• 2 does the same thing as the previous number, but it does so for other users.
You can use the umask command directly in the CLI to set the default permissions
for that session, or you can set the default permissions for each user in their
.bashrc or .profile file.
The syntax of the umask command is umask {number}.
Because it subtracts from the default (666), the umask command cannot force newly
created files to set the execute bit.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The chmod command has two different syntaxes for setting permissions: absolute
mode and symbolic mode. Both approaches provide the same results. Some
administrators find one approach more logical than the other, and it does not
really matter which is used. Sysadmins should understand and be able to use both
modes.
Absolute Mode
Absolute mode uses octal (base-8) numbers to specify permissions. Each
permission (r/w/x) has an associated number.
Absolute mode octal values:
• Read = 4
• Write = 2
• Execute = 1
By adding the octal numbers for the permissions you want to grant, you get
the overall permission number to assign to a directory or file. For example, full
permissions (read, write, and execute) are equivalent to 4 + 2 + 1, or 7. Read and
write permissions are equivalent to 4 + 2, or 6. Complete permissions are expressed
as a three-digit number, where each digit corresponds to the owner, the group, and
others, respectively.
In absolute mode, the syntax of the chmod command is chmod {number}
{file/directory name}
Three-digit and Four-digit Modes
When written in octal, numeric format, file permissions typically have three digits,
with each digit corresponding to the user, group, and others permissions. However,
file permissions may also be written with four digits, with the new, leading digit
signifying any advanced permissions to be defined (or 0, for none). For example,
the permissions for a non-executable file in Linux might be rw-rw-r--, or 664. This is
equivalent to the octal format of 0664.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Symbolic Mode
Symbolic mode enables you to set permissions using three components, namely:
• Permission contexts: u/g/o/a (a applies the permissions to all three contexts)
While sysadmins tend to prefer one mode or the other, it is worthwhile to understand
both modes. Documentation, online resources, and other administrators may provide
permissions settings using either mode, so you must be able to interpret whichever
mode is presented.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Configure Ownership
Although you’ve restricted access to your data by assigning permissions, you may
need to allow users to modify those permissions for certain files and directories.
This is where the concept of ownership comes into play.
As you’ve seen, the first permission context is the owner or user. In Linux,
ownership refers to the property by which a user can apply and modify the
permissions of a file or directory. By default, the owner of a file or directory is the
user who created that file or directory. Other than the superuser, no other user
can change the permissions of an object that is owned by someone else. While the
most common application of ownership is the changing of read, write, and execute
permission attributes, owners can also alter advanced and special permissions of
the objects they own.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The chown command is used to change the owner, the group, or both for a
file or directory. At times, you may wish for someone else to manage an object’s
permissions other than the user who created that object.
Changing the owner with chown. The file named “project1.txt” was owned by root, but is now
owned by student5 in this example.
Configure Attributes
Files can have one or more attributes set on them that define how the system
interacts with those files. These attributes go beyond typical permissions and
enable you to more granularly customize what the system is and is not allowed to
do with a file.
There are many such attributes. Some examples include:
• Only allow the file to be open for writing in append mode (i.e., don’t allow the file
to be overwritten).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The lsattr command displays current attribute settings, such as this immutable flag.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Thanks,
Ali
Date last updated 2024-04-25
Attribute Issues
Attribute problems are typically related to whether or not the attribute is set. Often
the solution is to either set or unset the value.
If the file can be deleted and should not be, set the immutable attribute with
chattr +i {filename}
If the file cannot be deleted and should be, unset the immutable attribute with
chattr -i {filename}
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Standard Linux Permissions
3
1. How does the principle of least privilege help mitigate threats and
mistakes?
3. Write the command by using symbolic mode that removes the read
permission from others for fileA without impacting other permissions.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 3B
Configure Special Linux Permissions
5
The standard read, write, and execute permissions are good enough in most
circumstances. However, there are additional permissions and attributes that you
can use to restrict access in a more specialized way. These permissions permit
users to run software as someone else or allow directory contents to inherit group
associations, providing default access control settings.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• Confirm the SGID permission is set correctly for directories to permit files
created in the directory to inherit the group association.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Special Linux Permissions
6
1. How would SGID benefit users when set on the /projects directory where
multiple users are members of the associated group and need access to
each other’s files in the directory?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 3C
Configure Access Control Lists
3
One weakness of standard Linux permissions is that only a single user, a single
group, and all others who are not that user or a member of that group can be
granted access. With standard permissions, a sysadmin cannot provide UserA with
rw- and UserB with r--. Nor can GroupA be given rwx to a directory while GroupB is
given r-x.
Access control lists, a feature of the filesystem, do permit the system to recognize
multiple identities and enforce different levels of access on those identities. In other
words, with ACLs the scenarios above are possible.
ACLs do not replace standard permissions, but rather they enhance them.
Everything you know about standard permissions still applies and is then
supplemented with additional ACL features.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The getfacl command output shows the file owner, group, and standard permissions.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The setfacl command uses the -m option to modify the access control list already in place.
ACL settings do not replace standard permissions. Standard permissions are still
enforced as described earlier in this Lesson, but they are further enhanced by ACL
capability.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
ACL Configuration
4
2. What commands are used to set ACL entries for USERA with rwx and
USERB with r-- for fileA?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Lesson 3
Summary
4
The identity of a user or a user’s membership in a particular group governs access to files and
directories. This access control, provided mainly by permissions, allows the possible viewing,
editing, or running of files. Additional controls, such as attributes, default permissions, and
access control lists, permit administrators to exercise more granular control over resources.
Guidelines
These best practices and guidelines are provided for your use as revision tools or as quick
references in your job role.
• Understand the principle of least privilege, which enforces the idea that users should be
given as little access to resources as necessary for them to do their jobs, with no additional
unneeded access.
• Absolute mode and symbolic mode provide the same information in different ways.
Absolute mode displays in octal numerals, while symbolic mode displays information using
operators.
• The immutable flag is an attribute of a file or directory that prevents it from being modified,
even by the root user.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
This Lesson covers the Filesystem Hierarchy Standard (FHS) that defines consistent
locations for data across the various Linux distributions and makes it easier for
users to navigate among directories to find and manage files and directories. Users
rely on an understanding of the FHS to perform file management tasks such as
creating and deleting resources. This Lesson covers the necessary commands to
perform these file operations. In addition, commands such as find and locate are
used to search for resources using many different parameters.
Lesson Objectives
In this lesson, you will:
• Understand the Linux file system.
• Locate files.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 4A
Understand the Linux File System
2
Standard Subdirectories
/bin - Stores essential command-line utilities and binaries. For example, the
/bin/ls is the binary for the ls command.
/boot - Stores the files necessary to boot the Linux operating system.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
/dev - Stores hardware and software device drivers. This directory maintains file
system entries that represent the devices connected to the system (for example,
the /dev/sda1 partition).
/etc - Stores basic configuration files. For example, the /etc/ samba/smb.
conf file stores Samba configuration data.
/home - Stores users’ home directories, including personal files.
/lib - Stores shared program libraries required by the kernel, command-line
utilities, and binaries.
/media - Stores mount points for removable media such as CD-ROMs and floppy
disks.
/mnt - This is the mount point for temporarily mounting file systems.
/opt - Stores optional files of large software packages. These packages normally
create a subdirectory bearing their name under the /opt directory and then place
their files in the subdirectory. For example, the /opt/nessus subdirectory
contains files for the Nessus vulnerability scanning program.
/proc - This is a virtual file system (VFS) that represents continually updated kernel
information to the user in a typical file format (for example, the /proc/mounts file).
/root - The home directory of the root user.
/sbin - Stores binaries that are used for completing the booting process and also
the ones that are used by the root user. For example, the /sbin/ifconfig file is
the binary for the ifconfig command that is used to manage network interfaces
on the system.
/sys - This is another VFS, and it primarily stores information about devices.
For example, /sys/block includes links to devices that are stored in various
subdirectories under the /sys/ devices/ location, which presents a hierarchy
of devices in the kernel.
/tmp - Stores temporary files that may be lost on system shutdown.
/usr - A read-only directory that stores small programs and files accessible to all
users.
The /usr directory contains some important subdirectories.
• /usr/bin - Includes executable programs that can be executed by all
users.
• /usr/local - Includes custom build applications that are stored here
by default.
• /usr/lib - Includes object libraries and internal binaries that are
needed by the executable programs.
• /usr/lib64 - Serves the same purpose as /usr/lib, except it is
meant only for 64-bit systems.
• /usr/share - Includes read-only architecture independent files. These
files can be shared among different architectures of an operating system.
/var - Stores variable files, or files that are expected to constantly change as
the system runs. Examples include log files, printer spools, and some networking
services’ configuration files.
The FHS is displayed by running the following command: ls / (recall that the
forward slash represents the root of the filesystem).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Key Directories
The critical directories to know from the FHS are /etc, /home, /root, and
/var/log.
/etc
The /etc directory contains files and subdirectories that hold configuration
information for the system and its services. It’s reasonable to assume that if you
need to adjust firewall rules, manage Secure Shell (SSH) access, or configure
software-management settings, you will need to work with files stored in /etc.
/home
The home directory contains a user’s personal files or files that are otherwise
specific to that user. The home directory is where you are placed when you log
in to the system. In Linux, by default, every user except the root user is assigned
a subdirectory in /home that corresponds to their user name. A user can create
subdirectories and files within this directory.
The home directory path is set to a variable named $HOME. When the string
$HOME is used, it references the path to the current user’s home directory, such as
/home/student-user. This text will often reference the home directory using
the $HOME variable. In many shells, including KornShell, C shell, and Bash, the tilde
character (~) represents your home directory.
/root
The home directory of the root user is /root. This is not to be confused with the
root directory (/), which is the top-most part of the file system hierarchy.
/var/log
Linux and installed services typically write to log files stored in the /var/log
directory. As a sysadmin investigating a system issue or auditing system access, you
will likely find the information required in log files stored at /var/log.
Familiarity with all of the directories is important, but the directories listed above
are more commonly used and accessed.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Absolute Path
The absolute path defines the directories to traverse, starting from the root of
the filesystem. Recall that the filesystem root is depicted using a forward slash
character (/). The absolute path is the complete path from the top of the directory
structure.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Suppose a user named student-user wants to find the widgets.txt file, which
is stored in the inventory subdirectory of the resources directory of their home
folder. The absolute path is:
/home/student-user/resources/inventory/widgets.txt
Relative Path
A relative path is dependent upon the user’s current location in the filesystem.
Using the above scenario, if the user is already in their home directory, then the
entire path is unneeded, and all that needs to be specified is the remainder of the
path from the current location. Assuming the user is in their home directory, the
relative path is:
resources/inventory/widgets.txt
In other words, relative to your current location, what’s the rest of the path?
Understand Inodes
An index node (inode) is an object that stores metadata about a file or directory
on a file system. This metadata can include time-based values such as when a file
was created and last modified, permission and ownership information, the block
locations of a file’s data on a storage device, and other miscellaneous information.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Use the ls -i command to list the inode numbers for files and directories.
Link Types
Using the ln command, you can create two types of links: hard and symbolic
(soft). Hard and symbolic links are a feature of the file system and are common in
most file systems supported by Linux. The ext2, ext3, ext4, and XFS file systems all
support hard and symbolic links.
The syntax of the ln command is ln [options] {target name}
{link name}
The ln command has various options. Some of these options include:
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
The Linux File System
3
3. A user submits a ticket regarding a file access issue. The first file,
projectA.txt, had a second hard link named my-project.txt. The same
data was available via either link. The user deleted the my-project.txt
file, and the data was still available via projectA.txt. The second file,
projectB.txt, had a sym link that pointed to the projectB.txt link. When
the projectB.txt link was deleted, the data was no longer available via
the sym link. The user wants to know why the first file is still available
after a link deletion but the second is not.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 4B
Use File Management Commands
4
Once the FHS is familiar, navigation among directories is much easier. Commands
such as pwd and cd allow users to understand where in the filesystem they are
and how to move to another location. Over the years, shortcuts have been added
to reduce the amount of typing at the command prompt. Users must manage
file operations—tasks such as moving, copying, creating, and deleting files and
directories—to manage their data correctly. Users must also be able to redirect
information into or out of files.
Navigate Directories
Directories store system configuration files, user data, log files, and other
resources. Linux users move from directory to directory by using various navigation
commands. Shortcuts make the navigation process quicker and simpler.
Knowing how to navigate through the directories allows Linux users to understand
specific directory navigation information such as:
• Where in the file system you are.
Navigation Shortcuts
Absolute and relative paths clearly delineate how to travel from one directory to
another, but over the decades, shortcuts have been established to make common
tasks easier.
The single dot character (represented by “.“) means “here“ or this directory. If your
present working directory is /home/USERNAME/resources, then that path
can be abbreviated as a dot: .
The term parent directory refers to whatever directory is above the current directory.
If your current location is /home/USERNAME/resources/inventory, then
the parent directory of inventory is resources. The parent directory of resources is
USERNAME. The parent directory is often referenced in commands (particularly with
cd), and it has been abbreviated as two dots (represented by “..“). In the above
scenario, the result of typing cd .. is to move to the resources directory. This is far
less typing than the absolute path of /Home/USERNAME/resources.
One of the most useful shortcuts is the tilde symbol (~). This abbreviation means
the home directory of the currently logged-in user. This is incredibly handy. A user
could type cd ~ instead of cd /home/USERNAME to jump to their home
directory. The ~ can be combined with many other commands covered below,
including copy and paste functions.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Using the cd command to return to your home directory is so common that even the ~
is strictly necessary. If you type cd with no argument (path), Bash assumes you wish to
return to your home directory.
Change Directories
The cd command moves your present working directory to another directory. For
example, if you type pwd and discover you are currently in your home directory,
then you could type cd /etc to move to the /etc directory, where many
configuration files are stored.
The cd command is the primary filesystem navigation command in Bash.
The syntax for the cd command is cd {path}
Note that either an absolute or relative path can be specified, depending on your
current location in the filesystem and the desired destination. To traverse the
filesystem to the inventory subdirectory in the resources directory of your home
directory, type the following:
cd /home/USERNAME/resources/inventory
Know Your Location
The critical piece of knowledge for navigation is to know your current location. In
Linux, this is referred to as the “present working directory,“ and the pwd command
displays that. The output is an absolute path from the root of the filesystem to the
directory the shell is currently focused on. The output from pwd may be used to
confirm your location before a backup job, a file transfer, or the deletion of files.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Output of the pwd command, showing that the user is in the /home/student directory.
Some Linux distributions are configured to display the name of the current directory
as part of the command prompt. Note that this is not the absolute path but merely the
name of the current directory. The prompt configuration can be changed.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Files where the first character is a dot (".") are hidden by Bash. This isn't a security
feature, but rather a way of keeping some files out of the user's way. Profile files are
often hidden. Use ls -a to display all files, including hidden files.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Commands used to create a project directory with three subdirectories, and use of the ls
command to display the directories.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
While copying a file creates a new file (a duplicate of the original), moving a file
typically places the file elsewhere in the filesystem. While the file itself may not
move from its location on the hard drive, pointers to the file will be updated with a
new location.
The mv command moves one or more files from one location to another. Like cp,
the mv syntax is to specify the source location first and the destination location
second.
The syntax for the mv command when moving a file is mv {source-
location} {destination-location}
For example, to move a fileA from your home directory to the Projects directory in
your home folder type:
$ mv fileA Projects
However, the mv command has a second function. It also serves as the rename
command in Bash. The syntax is the same, but the terms shift to original-filename
and new-filename. The syntax for the mv command when renaming a file is mv
{original-name} {new-name}
For example, to rename an existing file named “sales-project.txt“ to “marketing-
project.txt“, type:
$ mv sales-project.txt marketing-project.txt
In the project management scenario, the cp and mv commands are very useful for
organizing files.
First, you’ll copy the timeline.txt file to the resources directory:
$ cp timeline.txt resources/
Next, you move the newprojects.txt file to the drafts subdirectory:
$ mv newprojects.txt drafts/
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Using the cat command to confirm the newly created text exists in the newprojects text file.
There are many ways to display file contents besides using cat. A text or
configuration file may be opened with a text editor such as Vim or Nano, or
database resources may be accessed via database programs such as MariaDB.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Observe that to delete an empty directory, the command is rmdir, but to remove a
non-empty directory, the rm command is combined with the -R (recursive) option.
If the user is in their home directory, the entire project directory structure can be
deleted by using the rm command and the -R option. Again, be very certain about
what’s being deleted before running this command:
$ rm -fR project
Use the ls command to confirm the project directory and its contents are gone.
Files are not recovered from the CLI. There is no concept of a trash can from which to
recover deleted files. Be careful before using the rm and rmdir commands!
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Not all file contents fit on the display. When using cat to display a long file, only
the contents at the bottom of the file may fit on the monitor. The remainder scrolls
upward, off the screen. The less command breaks the output into pages that fit
on the screen and can be cycled through.
There are two ways of using less. The first is directly, as a primary command to
display file contents. An example is less long-file.txt. The second way to
use less is to better organize the output of other commands. For example, the
/etc directory has many subdirectories and files—too many to display on the screen
simultaneously. The output of a ls command can be redirected into less, which
allows pagination through the output of ls.
The syntax for this example is ls /etc | less
The pipe character | is discussed later in this Topic in the redirectors section.
The more command is similar to less. The primary difference between the two
commands is that with less, you can paginate up and down, while with more you
can only paginate downward.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The tail command also has some very practical options. Like head, the tail
command accepts the -n option to modify the number of displayed lines. In
addition, the -f (or follow) option updates the output whenever the bottom of the
file changes. This is useful for following log files, as the display will update whenever
the log file changes.
The basic syntax for the tail command is tail {filename}
To display the last 15 lines of the file, type tail -n 15 {filename}. However,
to follow a log file and see updated output whenever the log file changes, type
tail -f {filename}.
Commands such as less and tail display the specified file contents, regardless
of whether it contains the data administrators or users need. These commands
don’t search for particular information within the file, they simply display its existing
content.
The grep command is a pattern matcher, which is a complex way of saying it
searches for strings of characters within a data stream. For text files, grep is a find
utility that displays the specified string or search term in the output.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The syntax for these combinations can take on several different forms.
To list all files in the /etc directory that contain the string “net“ in either uppercase
or lowercase:
To list all files in the /home/student directory that contain the string “.txt“:
The pipe (|) character is covered in more detail in the next section of this lesson. The
ps command is also covered later in the course.
The egrep command is a modified version of grep and functions similarly from
the user perspective, but this utility is deprecated in favor of grep -E.
Redirectors
All of this management and manipulation of files is useful for more than just
looking at the results in a terminal. When you use the terminal, or log out entirely,
you’ll want to ensure that crucial information is stored in a file for later retrieval
and analysis. In addition, you’ll benefit from combining multiple commands in
conjunction, making your administrative duties more efficient and powerful.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Redirection
Operator Action Example Results
> Redirect the standard ls > file1.txt The output of the
output to a file. ls command is
redirected to a file
named file1.txt
>> Append the standard ls >> file1.txt The output of the
output to the end of ls command is
the destination file. appended to a file
named file1.txt
2> Redirect the standard ls file3.txt 2> The output will not
error message to a errorfile.txt be displayed on
file. the screen but is
redirected to a file
named errorfile.txt
2>> Append the standard ls file3.txt 2>> The output will not
error message to errorfile.txt be displayed on
the end of the the screen but is
destination file. appended to a file
named errorfile.txt
&> Redirect both the ls file1.txt file3.txt The output will not
standard output and &> errorfile.txt be displayed on
the standard error the screen but is
message to a file. redirected to a file
named errorfile.txt
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Redirection
Operator Action Example Results
< Read the input from mail user@ The myletter.txt
a file rather than address < file is taken as the
from the keyboard or myletter.txt input and attached
mouse. to the email
message.
<<{string} Provide input data cat <<EOF The cat command
from the current This is a here will use the rest
source, stopping document. of the lines in this
when a line EOF file as input. It will
containing the stop accepting
provided string that input when it
occurs. When placed reaches the string
in a script, this is EOF. This string
called a here can be named
document. anything you want.
The output of the
cat command
would therefore
be: This is a here
document.
Redirection operators, their syntax, and results.
Command Modifiers
Several command modifiers exist that allow users to run combinations of
commands. These commands differ from redirectors in that they manipulate the
command, not the output. Learning to use these effectively makes command-line
administration more efficient.
Background a Command
The single ampersand & causes the command to execute in the background.
Normally, when a command is executed, it consumes the shell until it completes.
Some commands, such as backup scripts, may take a very long time to finish, and
until the script is done, the user cannot type in additional commands or accomplish
additional tasks in that shell instance. If the command runs in the background,
however, it executes while the user continues to work within the shell. Such
commands are referred to as jobs.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Standard interaction with the Bash shell involves typing a command, executing the
command, and then typing another command for execution. Commands can be
written sequentially at a single command prompt to be processed in order. There
are multiple ways of doing this.
Piping is the process of combining the standard I/O streams of commands. It uses
the standard output of one command as the standard input for another command.
The output format of the first command should be compatible with the format that
the second command works with. The pipe operator (|) can be used with most
commands in Linux.
$ ls -l | grep audit
This command, which searches for files named “audit,“ is an example of using a
pipe. The standard output of the ls -l command is fed as standard input into the
grep audit command so that grep searches for the term within the directory
listing.
Chain Operator
Character Common Name Results
; semicolon If the semi-colon character is
placed between commands,
they are run one after another,
regardless of whether the
previous command completed
successfully.
&& logical AND If the logical AND is placed
between commands, the
following command is only run
after the previous command
completed successfully. Useful
for commands that rely on
each other’s success.
|| logical OR If the logical OR is placed
between commands, the
following command is only run
if the previous command fails.
! bang If the bang is placed in a
command expression, it
negates the expression. Used
as a way of stating “except
this.“
Many of these command chains and redirectors are used in scripts. Scripts are covered
in a later section.
These command modifiers do not use a typical or regular syntax. Instead, they are
used between commands to modify certain portions of the command expressions.
To run a series of commands one after another:
$ backupscript.sh;rmdir /projects;mkdir holidays.txt
To run a series of commands that rely on the success of the previous command:
$ backupscript.sh && rmdir /projects
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
File Management Commands
5
1. You have been assigned a helpdesk ticket to answer a user question. The
user is attempting to rename files but cannot find the rename command.
What command do you teach the user?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 4C
Find File Locations
4
Users and sysadmins may struggle to find older files managed by the system and
applications. The find command permits users to search for files based on specific
parameters such as name, ownership, permissions, and more. Users can use
commands such as which to discover where files were executed. Finally, many
file manipulation commands, such as sed, awk, wc, echo, and others, help
sysadmins find data available within files.
Search Commands
Even though the Linux filesystem is fairly well organized by the Filesystem Hierarchy
Standard, and users have file management options with mkdir, mv, and grep
commands, sometimes files are lost or their location is unknown. The primary
command for searching for files using parameters is the find command. The
locate command is an alternative that may make searching easier in a very large
filesystem. Finally, the which command displays where a command executable file
is stored.
Response Hi Joseph, you can search by the file name or by the file
owner using the find command.
Date last updated 2024-05-10
The find command searches the filesystem for files that match the given parame-
ters. These parameters might be file size, modification date, owner, or even permissions.
The find command is a powerful tool for managing files.
The syntax for find is more complicated than some other commands. The general
syntax is:
# find {where to search} {search criteria}
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
For example, to search for a file named “business-plan.txt“ in the projects directory
within your home directory, type:
# find -name business-plan.txt /home/USERNAME/
projects
The -name option specifies to search by name.
A search by permissions of the /home/student directory returns only those files with the
664 access level.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The updatedb command is used to both build and update a database of files
based on the /etc/updatedb.conf file. The database itself is located at
/var/lib/mlocate/mlocate.db. The configuration file can be set to
exclude particular directories that should not be indexed by the utility.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The tr Command
The tr command is used to translate a string of characters. It is predominantly
used to change the case of letters in a file. This command acts only on a stream of
characters and does not accept file names as arguments. You must use redirection
to actually change a file.
The syntax of the tr command is tr {character 1} {character 2}
where {character 1} is the character to be replaced.
The wc Command
The word count (wc) command is used to count the number of lines, words, and
characters in a text file. If multiple files are specified, then the command displays
the counts for each file and the total count for all files.
The syntax of the wc command is wc [options] {file-names}
The wc command provides various options that enable you to specify the nature of
the output.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• Tokyo,Japan
• London,United Kingdom
• Lima,Peru
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
File Location
5
1. You are conducting a security audit and need to document user access
to log files—specifically whether any files are world-readable or whether
any allow rwx access to all users. How can the find command be used in
such a project, and what specific command might you use?
3. A senior sysadmin suggests that commands such as sed, awk, and sort
are just as useful in automation as at the command-line. How might
commands such as these be used in automation?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Lesson 4
Summary
4
Guidelines
These best practices and guidelines are provided for your use as revision tools or as
quick references in your job role.
• Recognize file management commands
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
Linux users edit text files in day-to-day activities and when making system
configuration changes. Because a graphical user interface (GUI) is not always
available, visual menus are not always possible; text editing in Linux is very different
from text editing in other platforms. Text editors rely on various modes and
metakeys to enable functions such as save and copy/paste. It’s also essential to
ensure that files are backed up, protected, and that their integrity is guaranteed.
This Lesson focuses on Linux text editors such as Vim and archiving tools such
as tar.
Lesson Objectives
In this lesson, you will:
• Edit text files.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 5A
Edit Text Files
2
Text file editing is a basic but essential skill for Linux users. In Linux the graphical
user interface is optional, making text editing unique in Linux compared to other
operating systems. If the Linux system does not have a GUI, editors that rely on
mouse-driven menu options cannot be used. Sysadmins must use the keyboard
only to access all text editor program functions in this case. This Topic covers the
common Linux text editors that all users should be familiar with. It is critical to be
able to open, edit, save, and close files with any Linux editor.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Vim
The “Vi Improved,” or Vim editor, is installed on most distributions. It is extremely
powerful and a little complex, but the general functionality is straightforward. Vim
has three modes. When switching modes, the keyboard will react differently. For
example, pressing the w key in one mode inserts the “w” character in the text of the
file. In a different mode, however, the w key writes or saves changes to the file.
The older version of Vim is named vi (pronounced “vee-eye”), and it is not often used
on modern Linux systems. Frequently, when a user types vi to invoke the editor, the
Vim editor is automatically invoked instead. The vi editor is more limited, but its basic
functionality is the same as that of Vim.
To make working with Vim easier, begin by memorizing what the three primary
modes do and how to switch among them.
Both Command Mode and Execute Mode manipulate the file by doing an action
such as creating or saving a file, while Insert Mode actually manages the text in a
file (writing or deleting words). The default mode you see upon opening Vim is the
Command mode.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
There is a fourth Vim mode named Visual, which allows the user to select or highlight
text for copying, deleting, and other tasks. Visual mode is not covered in this course.
There are many additional commands in Command mode, but for the purpose of
learning the editor, only some commands have been documented here.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
:q Quit Vim.
Observe that ZZ in Command mode and :wq in Execute mode do the same thing.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Users may find it useful to compare two files or may need to transfer content
between the files. In this case, it’s useful to have the files open side-by-side (vertical
split). The user can glance back and forth between the two file versions, making
whatever edits are desired.
Split Vim into two or more vertical windows by typing Ctrl+w and then v.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
To cycle from one window to the next, type Ctrl+w and then w. You can also use
Ctrl+w with the four normal directional keys mapped in Vim.
nano
Nano is a simpler (but less powerful) text editor than Vim. As such, many users
and administrators prefer nano for basic editing functions. Nano does not use the
concept of modes. Instead, pressing the Ctrl key on the keyboard along with
various character keys issues commands to the program.
The nano command invokes the editor. If an existing file is specified, the editor
opens it. If no file is specified, nano creates a new file for editing.
Note that the name of the text editor is “nano," with no capitalization.
Shortcuts in nano
When nano opens a new or existing file, no further action is needed to begin editing.
Simply pressing keys on the keyboard begins the process of entering text. You can
move the cursor through the file using the arrow keys. Most commands are issued
to nano by using the Ctrl key; however, other keys, such as Alt and Tab, may
be used for some functions.
Nano is quite powerful and contains many keystroke shortcuts in addition to those
mentioned here. Use the arrow keys, Page Up, Page Down, and Home keys to
navigate within a document.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Copying parts of text on a line requires you to “mark” the text you want to copy with
the Ctrl+^ shortcut. You then navigate your cursor to highlight the text you want
to copy. Pressing Alt+^ copies the marked/highlighted text, and Ctrl+U pastes
it. You can cut the current line by using Ctrl+K.
Gedit
The gedit text editor is the default text editor used in GNOME desktop
environments and is a member of the GNU Project. Unlike Vim and nano, gedit
has a GUI with a typical menu-based design that makes it easy to work with. It also
has features such as syntax highlighting and spell checking and can be customized
through plugins. While not as powerful as Vim, gedit may still be useful in systems
that have a desktop environment installed.
Although you can launch gedit from the desktop, you can also use the CLI with the
gedit command. The syntax is similar to vim and nano—typing the editor name
with no argument opens a new file, whereas providing a file name as an argument
either opens an existing file or creates a new one with that name.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Text Files
3
1. A user contacts you and wants an easier text editor to use than Vim.
There is no GUI installed on the system in question. What text editor do
you suggest and why?
3. Why are text editors more important on Linux systems than on other
systems?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 5B
Manage Text Files
4
Both user and system files must be protected. It is common to back up user files
in case of hard disk failure. It is also common to back up system configuration files
before making major changes. Doing so offers the ability to restore the original
configuration file if something goes wrong with new changes. The tar utility is
often used to back up or compress files. In addition, applications or other data
files available on the web are commonly bundled with tar for ease of access. After
downloading such files, verify their integrity to ensure the files are legitimate,
complete, and uncorrupted.
Thanks,
Ali S
Date last updated 2024-05-14
When using tar to work with a group of files, the primary phases are:
1. Create a tarball (bundle).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The syntax for the tar command is tar [options] {file1, file2,
file3...}
Some common options for tar are:
Thanks,
Ali S
Response Sure thing, Ali. Please take a look at the attached walkthrough
using the -c and -x options with tar
Using the example in the helpdesk ticket, the following list is a demonstration of the
tar utility. This series of commands will manage user Ali Selassi’s proposed archive
consisting of file1, file2, file3, and file4.
1. Create a new tarball:
tar -cvf filearchive.tar file1 file2 file3
2. Display the files in the archive:
tar -tf filearchive.tar
3. Add a file to the archive:
tar -rf filearchive.tar file4
4. Extract the files from the archive (this is equivalent to a restore process in a
backup/restore scenario).
tar -xf filearchive.tar
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The full CLI output of the different steps of the tar utility.
Keep in mind that as a user, you will frequently interact with tar archives as part
of a download process. Experience with tar makes both file management and
download processes easier.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Compress Files
Compression is a procedure in which data is encoded to reduce the amount of
bits that are used to represent that data. The compression process can significantly
reduce the size of a file or collection of files in order to make the storage and
transfer of data more efficient. Although the file takes up less space, it still contains
the requisite information so that only redundant data is removed (lossless
compression) or so that only noncritical data is lost (lossy compression).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Compressing with xz
The xz command is a data compression utility, similar to gzip, that reduces the
size of selected files and manages files in the .xz file format. The xz command has
several options.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
For compression speed, tests tend to show that gzip is slightly faster than bzip2,
and both are significantly faster than xz when the applied compression level
increases. For decompression speed, gzip tends to be the fastest again, with xz and
bzip2 as second and third fastest, respectively. When it comes to compression ratio,
xz tends to perform the best, followed by bzip2, with gzip having the worst ratio.
Ultimately, consider using:
• gzip—if you just care about compressing and decompressing files as fast as
possible and are less concerned with storage space.
Thanks,
Ali S
Response Yes, you can combine the tar utility with a compression tool
like gzip or bzip2. I’ve sent over a tutorial on how to use it.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
You'll use the md5sum command to check file integrity in Lesson 12.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Text File Management
5
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Lesson 5
Summary
3
Linux users must be able to open, edit, save, and close text files using various text
editors, especially Vim and Nano. Not only is user data held in text files, but system
settings are also stored in these files and must be edited to make configuration
changes. Furthermore, user data should be backed up or may need to be available
for transfer to other systems. Archiving tools like tar play an important role in file
management. The text editing skills covered in this Lesson are critical to all facets of
Linux system use.
Guidelines
These best practices and guidelines are provided for your use as revision tools or as
quick references in your job role.
• Make a copy of configuration files before making changes.
• Be aware that various distributions may offer one or the other (or both)
editors.
• Recognize that tar is used with backup/restore processes and file downloads.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
Maintaining software on Linux systems is usually accomplished with package
managers. Package managers deploy, update, remove, and report on applications.
However, due to the open-source nature of the Linux environment, software
applications are often compiled from source code—something that is not common
with other operating systems.
Because there are many different Linux distributions available, there are many
package managers. As a general rule, most distributions either originated with Red
Hat Linux (and therefore use the Red Hat package managers) or Debian Linux (and
therefore use the Debian package managers).
This Lesson covers several common package managers, the process of acquiring
and compiling software, and running software in a sandbox for security.
Lesson Objectives
In this lesson, you will:
• Understand software management.
• Acquire software.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 6A
Understand Software Management
2
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
• You can learn how the software is constructed, discover bugs, and contribute
improvements.
To effectively use Linux systems, users must be capable of both managing software
packages and compiling software from source code.
Software Packages
Linux software is often distributed in packages that contain everything needed
to run the software—executables, configuration files, supporting resources, and
documentation. Therefore, the process of installing, maintaining, and removing
software is referred to as package management, and the tools required to maintain
software are package managers.
One critical component of package managers is the ability to report software
information. Package managers track software installation information in a
database that can be queried. Such queries might include a list of all installed
software, a list of software versions, or specific information on a single software
package. The ability to query the package manager’s installation database provides
sysadmins with essential information on the system and its applications.
Querying a package manager. In step 1, a user at a terminal uses the # apt show nmap
command to find out more information about nmap. The package manager calls out to the nmap
package (step 2), which provides information back (step 3). Finally, the package manager displays
the nmap details at the terminal (step 4). (Images © 123RF.com.)
A second critical part of package management is the source from which package
managers pull the software. A storage location for software packages is referred
to as a repository. Sysadmins control what repositories are available to package
managers.
Lesson 6: Managing Software | Topic 6A
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
In an earlier lesson, the Filesystem Hierarchy Standard (FHS) was noted as being a
consistent way of organizing Linux directories. Such organization allows package
managers to exist. Package managers place configuration files in /etc, log files in
/var/log, documentation in /usr/share, and so on because those locations
are standardized across distributions.
Each package manager has its own related commands, and those commands cover
the three major parts of the software lifecycle (installation, maintenance,
and removal). Some basic examples are:
• $ rpm -i {software-package}
• $ rpm -U {software-package}
• $ rpm -e {software-package}
• $ apt install {software-package}
• $ apt upgrade {software-package}
• $ apt remove {software-package}
When discussing package managers, note that the name of the package manager is
usually shown in uppercase letters as an acronym. The associated commands are
displayed in lowercase letters because they would be typed at the command prompt.
So the RPM package manager includes the rpm command (to install .rpm files).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The commands for these specific package managers are covered in detail later in
this Lesson.
Software Dependencies
Package managers make software maintenance much more straightforward. One
challenging part of handling Linux software, however, is dependencies. Some
software packages rely on other packages to function. If the software an application
depends on is not installed, it will either fail entirely or not work as intended.
Early package managers required administrators to manually provide the
dependent applications before installing the primary package. Modern package
managers such as YUM, DNF, and APT install supporting applications automatically,
ensuring that a single installation command provides the user with everything
needed to run the program.
Periodically, package dependency issues still arise, so recognize the problem
(failed dependency) and the solution (install the packages required for the primary
application to run).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Software Management
3
3. Why is compiling software more common with Linux systems than with
other operating systems?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 6B
Manage RPM Software Packages and
Repositories
4
Many common Linux distributions are derived from Red Hat Linux. Common
components of most of these distributions are package management and software
package format. Packages for Red Hat–derived distributions use the .rpm file
extension. There are three common package managers: RPM, YUM, and DNF.
Common examples of Red Hat–derived distributions include Red Hat Enterprise
Linux and Fedora.
RPM Commands
The use of Red Hat package managers is primarily focused on installing, updating,
upgrading, and uninstalling software packages. These packages have a .rpm
file extension. RPM files are pre-compiled and usually include all supporting
documentation and configuration files. The idea is that a single package contains
the entire application.
Observe that in this Lesson, “Red Hat” usually refers to that Linux distribution in the
context of a parent distro from which many other Linux distros are derived. RPM-
oriented package managers are not necessarily developed or maintained by Red Hat
itself.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Here is an example of using RPM to manage a software package that installs the Z
shell (assume the zsh.rpm package has been downloaded to the system, and the
present working directory is the download location).
This command installs the software in verbose mode and provides a progress bar
to track the installation process:
$ rpm -ivh zsh.rpm
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
This command does two things. It will update the package on a system that already
has the package installed. It will also install the package on a system that does not
currently have it. To accomplish either of these, type:
$ rpm -Uvh zsh.rpm
This command will update an existing package, if installed, but it will not install the
package if it is not already present:
$ rpm -Fvh zsh.rpm
The importance of upgrade (-U) versus freshen (-F) may be difficult to visualize
when thinking of a single system. On a single computer, you’re likely to know
whether the specific software package is present. However, the real importance of
these two options is clear when the rpm command is used in scripts that maintain
software on many systems. A single script may be used to manage software on
20 systems, some of which are supposed to have the software and some of which
are not.
Removing the zsh.rpm package is just as simple. The progress bar and verbose
options may be used to track the removal process as well:
$ rpm -evh zsh.rpm
The actual zsh.rpm package name will, in reality, be much longer and include
information about the version and architecture for which the package is designed. The
package name used here has been shortened for these examples.
YUM Commands
The yum command works with .rpm packages and is the next evolution of the
RPM package manager. It features a more intuitive syntax and the ability to handle
software dependencies more elegantly.
The syntax for the yum command is yum [options] [subcommand]
{package-name}
There are many common subcommands for the yum command, but the key
examples are presented in the table below.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The command to check for updates for all installed packages is:
$ yum check-update
This may generate a lot of output. If there is a specific package that you want to
check for updates, add the package name as an argument:
$ yum check-update {package-name}
To remove software by using YUM, type:
$ yum remove {package-name}
The yum update command, with no arguments, updates all software packages on
the system.
DNF Commands
The DNF package manager is the most advanced of the three software-
management solutions and includes many practical features and updates. It is the
assumed package manager on modern Red Hat–derived distributions, though YUM
and RPM may also be available on those systems.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The DNF package manager recognizes package groups. Package groups are logical
collections of related packages. If an administrator wanted to install all software
for managing virtual machines, each individual package could be identified,
downloaded, and installed. However, virtualization software is collected into a single
package group to be installed as a unit:
$ dnf group install virtualization
It is also possible to identify the group by using the @ symbol in the following command:
dnf install @virtualization. Note that this example uses the
standard dnf install syntax.
Which to Use?
When working with Red Hat–derived distributions, assume the use of the dnf
command to manage packages. Sometimes the rpm command is useful for a single
quick installation when no dependencies are involved. While DNF is the standard,
YUM is still very common.
It’s a good idea to be able to administer basic package management by using all three
commands.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The output from the rpm -qa command is likely to be immense, so it may be a
good idea to redirect the results into a text file.
The syntax to display information about a specified package is:
$ rpm - qi {package-name}
Results of the rpm -qi command showing information about the nano package.
The YUM and DNF package managers provide the same functionality.
There may be instances when a user, administrator, security auditor, or other
individual needs to know exactly what software is installed on a specified Linux server.
Thanks,
KG
Date last updated 2024-04-11
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
It’s likely that the output of these commands will be redirected into a text file to be
provided to the requesting user.
Observe that the syntax for YUM and DNF is very similar.
Why Repositories?
Repositories, often called “repos,” are software storage locations that package
managers can check when installing or updating software. Many repositories exist,
and not all are trusted. In some cases, malware might be present, but there are
other reasons that repositories may not be trusted.
Possible reasons for sysadmins to control available repositories include:
• Software version control: Some repositories may release beta or unapproved
software that production servers should not use.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The yum command includes some additional subcommands for viewing and using
repositories:
• repolist —Used to see all available repositories.
• makecache —Used to locally cache information about available repositories.
• clean all —Used to clear out-of-date cache information.
Global settings, such as where to find repository configuration files, log file
locations, and cache information, are stored in the /etc/yum.conf file. The
difference between this file and the repository configuration files is that this file
defines how the YUM package manager works, while the repository files define
where the package manager searches for packages.
The DNF package manager uses a configuration file stored at /etc/dnf/dnf.
conf to define its global settings. These are the same types of global settings
as defined for YUM. DNF also looks to the /etc/yum.repos.d directory for
repository reference files.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Services are covered later, but recall that for configuration file changes to be
implemented, the sysadmin must restart services.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
RPM Software Packages and
Repositories
5
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 6C
Manage Debian-based Software
Packages and Repositories
4
While the previous section covered the Red Hat approach to package management
and this section covers the Debian package-management system, it’s worth noting
that package management concepts remain constant. Both approaches manage the
software through its entire lifecycle, from installation, to update, to removal.
Many Linux distributions trace their lineage back to Debian, one of the oldest
distros. The most widely used distros include Debian, Ubuntu, Linux Mint, and Kali
Linux, though there are many other examples. Debian’s common dpkg package-
management system has been replaced by the apt command.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Update Packages
As part of software life-cycle management, APT can update installed software.
Software updates are usually a two-step process with APT. The first step is for APT
to discover updated packages in the configured repositories. The second step is to
actually install the updates.
First, update APT with information about newer versions of available software
as compared to the packages already installed. This does not update the actual
software. The command is:
$ apt update
Once APT is aware of available software, it checks that software against the current
installations. By using the upgrade subcommand, administrators can upgrade
any or all packages to current versions. The apt command for upgrading is:
$ apt upgrade {package-name}
Lesson 6: Managing Software | Topic 6C
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
If no argument is specified with apt upgrade, then all packages are upgraded
to the latest version.
The dpkg package manager has mostly been replaced by APT in much the same way
that the RPM manager has been replaced by YUM and DNF.
Output of the sudo apt show command displaying details of the nmap package.
Sometimes a piece of software is unfamiliar to the user, and the ability to see
the vendor, the install date, and the description of the software makes it easier
to understand why the software is installed (and potentially whether it can be
removed).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Debian Software Packages and
Repositories
5
1. What are the two steps for upgrading software with apt?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 6D
Compile from Source Code
3
Package managers are efficient and convenient forms of acquiring and installing
software. However, there may be times when you want to build software from
source code—after all, most Linux software is open source.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The Bash shell only checks for executable files such as programs and scripts in specific
locations. Usually, a user’s home directory is not one of these locations (nor should it
be, for security reasons). However, it’s as if the software the user is compiling has been
downloaded to the home directory. The ./ instruction tells Bash to “check here” at the
present location for the specified executable file.
The second step is the actual compile process. In most cases, once the makefile is
created, simply issuing the make command without arguments will compile the
application. The make command automatically looks for the makefile in the current
directory. You can, however, issue make with various options.
A makefile is a file that contains instructions used by a compiler to build a
program from source code. These instructions typically define the resources that
the program depends on in order to function properly, as well as any additional
directives as defined by the developer.
Once the compile process completes, it’s time to install the software. This is the final
step in the process. The make install command installs the program, placing
the appropriate files in the appropriate locations (configuration files, log files, and
so on). The software is now available and ready to run.
Compile Software
When an administrator downloads software, there is a common process to build
the executable file called software compilation.
1. Unpack the download, typically using tar and/or gzip commands.
2. Change into the directory that gets created as part of the unpacking process.
4. Use the make command to compile the application using the information
stored in the makefile. Note that this usually requires root privileges.
5. Use the make install command to install the resulting binaries (the
application).
Many developers will provide instructions and options that may modify this process
somewhat. Specifically, there may be options or modifications that can be made
to the makefile before the make command is run to optimize the software for
the system or the user’s needs. It is important to review any README or other
instruction files that are downloaded as part of the software.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Source Code
4
2. Why is it more common to compile software with Linux than with other
operating systems?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 6E
Acquire Software
3
Repositories are not the only way to download and install software. Several other
tools are used in this topic to enable you to acquire the files necessary to use
software. These tools provide alternate ways of downloading files from web servers
and managing compression.
You can also search open-source hosting sites, such as GitHub®, for software.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
While there are many options for the curl command, common examples include:
Recall that tar was covered in an earlier lesson. It is covered here only in the context
of downloading software or source code from the Internet.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Software Acquisition
4
1. How can wget or curl be used to download files from the Internet in a
way that a web browser can not be used?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 6F
Run Software in a Sandbox
3
Typically, software and the operating system share resources and can affect each
other. This design creates potential security threats where one piece of software
can impact the entire system. Isolating software with sandboxing helps mitigate this
potential vulnerability.
Why Sandbox?
One of today’s greatest cybersecurity threats is ransomware. Malware may enter
a network environment via poorly tested or poorly written software, which then
gains access to user data. One way of mitigating this threat is for end-users to run
software in a sandbox environment.
A sandbox is a virtual machine or container that isolates the application from the
rest of the system and mediates communication between hardware resources and
the program. Such a design is often more secure than an operating system and
many applications sharing unmediated access to system resources. Developers
testing new software and cybersecurity professionals examining malware code are
among those who benefit from sandbox solutions.
Sandboxing differs from an older practice known as chroot. By using chroot,
administrators told the application that a given directory was the “root” or highest
point in the filesystem. Since the application was unaware of any directories above
this root, it couldn’t access the rest of the system. In some cases, this practice is still
appropriate today.
Sandboxing relies on applications to enforce boundaries, while chroot uses the
filesystem.
Sandbox Software
The following three package managers distribute and run software in a sandboxed
environment: Snaps, Flatpak, and AppImage.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Snaps
Canonical, the group responsible for Ubuntu Linux, uses the Snap package
manager to deploy and maintain software. The software is distributed as “Snaps”
and managed by the snapd daemon and the snap command. Each Snap is self-
contained with everything necessary to run included in the package. The packages
run in an isolated sandbox environment.
Once Snap is installed, packages can be downloaded from the Snap Store. After
that, application management is similar to that of other package managers:
$ snap install {software-package}
$ snap info {software-package}
$ snap list
$ snap remove {software-package}
Flatpak
Another package manager that runs applications in a sandbox is Flatpak. Using
the flatpak command, users can manage software on a system-wide or per-user
basis. Repositories—called “remotes”—store software distributed as flatpaks.
To install {software-package} from a remote named org.demo.sample,
type:
$ flatpak install {software-package} org.demo.
sample
AppImage
AppImage offers self-contained applications that run consistently on Linux systems
in a sandboxed environment. AppImage applications are distributed and managed
as a single file containing everything needed to run the application. Application
developers decide what distributions to support and then ensure that all needed
dependencies and libraries are included with the AppImage file.
AppImage applications are standalone components, and while they can be installed
in the traditional manner, they do not have to be. They can be accessed and run as
mounted drives.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Software in Sandbox
4
2. True or False? The original practice, known as chroot, that predates the
current method of sandboxing, is deprecated and no longer used.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Lesson 6
Summary
3
Red Hat and Debian package managers handle software maintenance for most
Linux distributions. There are also package managers that run applications in
isolated sandbox environments for ease of maintenance and security. Some
software is acquired in the form of source code, which allows users to customize
the application for specific features or hardware before compiling it. This process is
uncommon on platforms that use proprietary software but is more prevalent with
open-source code.
Guidelines
These best practices and guidelines are provided for your use as revision tools or as
quick references in your job role.
• Recognize which package managers are used by which distributions:
• rpm, yum, and dnf commands are usually used by Red Hat–derived
distributions.
• Be able to install, update, and remove software by using the rpm, yum, dnf,
and apt commands.
• Be able to query packages by using the rpm, yum, dnf, and apt commands.
• Be able to configure software repositories for the RPM, YUM, DNF, and APT
package managers.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
Storage management involves planning, understanding hardware, and using many
tools to accomplish storage goals. Sysadmins must know what types of hardware
are available and what type will work with the Linux system in question. They must
also know the performance requirements for data stored on any drives. Direct-
attached storage disks are managed as single entities with traditional partitions or
can be combined into logical volumes for more flexible storage space. Network-
based storage, such as NAS, SAN, and cloud technologies, may also be used. Finally,
sysadmins must understand the available Linux tools for troubleshooting and
performance monitoring.
Lesson Objectives
In this lesson, you will:
• Understand storage.
• Deploy storage.
• Troubleshoot storage.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 7A
Understand Storage
2
Storage Concepts
Storage capacity for user data, databases, logs, configuration files, operating
system and application executables, and other resources are of major concern to
sysadmins. Capacity is only one part of the issue, however. Related is the ability
to maintain cost-effective, reliable, and secure storage media that is accessible to
users and services.
The storage concepts in this section compare various storage technologies and
configuration options that inform the choices that Linux administrators make for
storage.
Cloud-based storage is a growing online service and has become a critical part of much
storage management. General cloud concepts are covered in a later Lesson.
• Solid-state disk (SSD): Flash memory storage, usually expensive, fast, small, and
shorter -lived
• Universal Serial Bus (USB): Connection protocol for various external devices,
including storage drives
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
File storage is the common approach to storing data on local drives, with data
organized into discrete files stored in directories. These directories are organized
in a hierarchical structure. Data is easy to retrieve and change, but file storage does
not scale well: it must be scaled out rather than scaled up.
Block storage is a good choice for large quantities of unstructured data
stored across multiple platforms and storage types. Commonly used with SAN
technologies, it is quick, reliable, and efficient.
Object storage is cost-effective and very scalable but best for static data because
it’s difficult to modify data once written. This makes it particularly unsuited for
databases.
A typical Linux server will rely on file storage by using one or more internal drives
that use a standard filesystem to organize and retrieve data.
Storage Interfaces
Internal storage devices use different protocols and interfaces to communicate
with the rest of the system. These designs vary by speed, communication type, and
scalability. Various solid-state drives, hard disk drives, and flash storage use these
interfaces.
Serial Advanced Technology Attachment (SATA) is a common inexpensive
storage media that uses a serial connection to communicate with the motherboard.
This is the slowest of the three options discussed here but the least expensive. It is
a good all-around solution for most systems.
Small Computer Systems Interface (SCSI) tends to be expensive with less capacity
than most SATA drives but makes up for this by being very fast. For servers, RPMs
of 10,000 and 15,000 are common, while desktop and laptop drives might spin
at 7,200 or 5,400 RPMs. Multiple SCSI drives can be attached to the same chain,
making their scalability far more flexible than SATA.
Serial Attached SCSI (SAS) is an upgraded SCSI design with larger capacities, faster
transfer speeds, and greater flexibility. However, these benefits come with a higher
cost. The cost may be well worth it for mission-critical servers.
Linux administrators must balance speed, capacity, and cost when deciding which
of the standards is appropriate for a new Linux workstation or server.
Types of Partitions
Storage drives are divided into partitions for more efficient and organized use.
On a Linux server, these partitions may store different types of operating systems,
applications, or data files. There are two ways of tracking these divisions of the
drive: a Master Boot Record (MBR) or a GUID Partition Table (GPT). These two
entries identify where on the physical drive the partition divisions are located.
The MBR is the traditional choice, but it is limited. Many server administrators use
the GPT because it recognizes more partitions (divisions) of the drive, which is
important for larger-capacity storage devices.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The master boot record (MBR) is the first physical sector on a storage drive and a
type of partition structure. The MBR boot sector contains the boot loader that loads
the operating system into memory. It also contains the partition table of the storage
drive. MBR determines what sectors are available to each partition, as well as which
partitions are bootable and which are not.
For many years, MBR was the dominant partition structure used in Linux and other
operating systems. However, it has three major disadvantages:
• The maximum storage space of an MBR-partitioned drive is two terabytes.
• The boot data is stored all in one sector, which increases the risk of corruption.
The GUID Partition Table (GPT) is a successor to MBR that makes up for the latter’s
shortcomings. Like MBR, it is a partition structure, but it employs a more modern
design and is part of the UEFI standard. Every partition on a drive is assigned a
globally unique identifier—a GUID—to distinguish it from every other partition on
(theoretically) every drive.
The storage space and partition number maximums are so large that they are not
currently achievable, and any limitations are going to be imposed by the file system
type or operating system kernel, rather than GPT itself. GPT also has the advantage
of storing its boot data in multiple locations on a drive to enhance redundancy.
If the primary location is corrupted, GPT can leverage one of the other copies to
restore the boot data.
Whenever possible, partitioning a drive with GPT is preferable to MBR.
The use of FUSE is not inherent in Linux administration. The key factor here is that Linux
is capable of this feature, which provides a great deal of flexibility.
RAID Arrays
A standalone storage disk is a single point of failure. When the disk fails, the data on
it is likely to become inaccessible. In addition, individual disks have limited transfer
speeds and storage capacity.
Storage disks may be organized into RAID arrays to mitigate the limitations of
standalone drives. A Redundant Array of Inexpensive Disks (RAID) combines
storage disks together. Depending on the configuration, the result may be greater
capacity, greater fault tolerance, and increased speed.
Two different designs manage RAID arrays: hardware and software. Hardware RAID
uses a dedicated controller attached to the motherboard. This is usually faster,
more flexible, and more expensive. Software RAID relies on the operating system
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
to manage the storage, which is usually slower, less flexible, and less expensive
(because no special hardware is required).
In a server, hardware RAID is almost always preferred.
Various RAID standards exist and are primarily identified by number, but three
of the most common are RAID 0, RAID 1, and RAID 5. Each has advantages and
disadvantages to consider.
• The total number of blocks (in kilobytes) that are being used by the user/group
on a file system.
• The total number of inodes that have been used on a file system by the
user/group.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
3
Storage Concepts
Answer the following questions:
1. Explain the key differences between MBR and GPT partition tables.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 7B
Deploy Storage
4
To confirm the proper disk type is installed and compatibility with the storage
controllers, type:
# hwinfo --short --storage
Add the --short option to focus on general information about the specified device.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The hwinfo tool is also used to gather CPU, memory, Bluetooth, and network device
data. It is covered in more detail in the next Lesson.
Output from the lsblk /dev/sda command displaying capacity and partition
information.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The sd string (which used to stand for “SCSI disk” but now means “storage disk”) is
followed by a letter defining the drive’s location. Any numbers that follow reference
a partition, if any have been created.
So, the first storage disk in the system is displayed as /dev/sda. In this example,
a second disk is added to the system, so it is labeled as /dev/sdb.
Partition Design
It is common to dedicate specific partitions to certain FHS directories. For example,
it may be useful to place the /boot directory on its own partition. In addition, the
/home and /var directories may also be mounted to dedicated partitions. The
reason these two directories are often attached to dedicated partitions is that they
are particularly at risk for filling up. Users may create or download a great many
files to their home directories, or log files stored at /var/log may suddenly
increase in size due to failing applications or services. If the partition where the root
of the filesystem (/) resides fills, the system will crash. To prevent this, directories
such as /home and /var, which are at risk for filling, are placed on separate
partitions from /.
Linux also commonly dedicates a partition to virtual memory. Using a process called
“swapping,” Linux moves data between memory and the hard drive if the memory is
in danger of being fully consumed. A dedicated swap partition is often set aside that
is 1.5 times the quantity of the total RAM.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
To create partitions on the newly installed storage disk by using fdisk, use the
following steps:
1. Type fdisk /dev/sdb to edit the partition table for the second (new)
disk.
7. Type p to display the new partition information (note that no changes have
been made at this point).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
8. Type w to write (save) the changes to the partition table and exit fdisk.
rm Delete a partition.
To create partitions on the newly installed storage disk by using parted (and only
if you haven’t done so via fdisk), follow this process:
1. Type parted /dev/sdb to edit the partition table for the second (new)
disk.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
5. Type quit to write changes to the partition table and exit parted.
GNU Parted is a critical tool for any sysadmin. The utility recognizes and can resize
New Technology Filing System (NTFS) partitions, which are commonly found on
Windows Servers.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The output of the blkid command displaying the UUID for the /dev/sda1 partition.
The ext toolset enables a variety of administrative tasks to manage the filesystem. These
tasks are covered in a later section.
To format the new partitions, use the mkfs command. The command expects to
know the desired filesystem and a target partition.
To place the ext4 filesystem on the second partition on the new disk, type:
# mkfs.ext4 /dev/sdb2
The mkfs also recognizes a slightly different syntax. The following command
achieves the same result as the example command above:
mkfs -t ext4 /dev/sdb2
The mkfs command overwrites existing data on the partition, making that data
difficult or impossible to recover. Use it carefully!
To format the second partition on the new disk with the XFS filesystem, use the
mkfs command but with a different option:
# mkfs.xfs /dev/sdb2
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
XFS Tools are available to help manage the filesystem. Management tasks include
resizing the filesystem and gathering information. The tools are covered in a later
section.
Observe that umount is a slightly confusing way of unmounting. The command and
the concept are not spelled the same.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Note that the syntax is similar to the mv or cp commands—from here to there. The
command could be interpreted as “attach the partition found at /dev/sdb2 to the
directory found at /projects.”
To test that the storage area is attached and usable, use the du and df commands.
External storage devices such as USB drives are mounted similarly. The path to the
device is specified along with the desired mount point. For removable drives, this
mount point is often a subdirectory of the /mount directory.
Assuming the USB drive is identified as sdc, use the following command to mount
the USB storage device:
$ mount /dev/sdc /media/usb
Sometimes there is confusion with the use of the word “filesystem.” One use involves
the format of a partition used to store data, such as ext4 or XFS. Another useinvolves
the organization of directories in a logical tree, such as the Linux Filesystem Hierarchy
Standard.
Output from the du -h /boot command displaying the size of each file in the directory.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The result shows files and directories consuming space in the /projects
directory. Add the -h option to display the capacity in a human-friendly format;
otherwise, the output will display in bytes, which may be difficult to translate.
Next, check the storage space in use by using the df command. The syntax is the
same as for du, with the command being: df -h /projects
Output from the df -h /dev/sda1 command displaying the size of the partition.
To see a summary of space in use on all disks, type df without any arguments.
Both utilities are important to troubleshooting, testing, and capacity planning.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
2. Confirm that Linux sees the storage device by using tools such as hwinfo,
lsblk, and lsscsi.
3. Partition the drive with fdisk or parted.
5. Manually test mount the storage capacity to a mount point by using the
mount command.
6. Ensure the storage space is usable by copying actual data to the location
using cp.
Manage Filesystems
Filesystems such as ext4 and XFS may require periodic maintenance. You may
wish to relabel the filesystem to something more descriptive or configure tunable
performance parameters. The filesystem may need to be expanded or shrunk
when using tools like parted to resize partitions. Here are several ext4 filesystem
management tools.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
XFS Tools
There are many tools that enable you to work with the XFS file system.
xfs_growfs Expand the XFS file system to fill the drive size.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The three levels of the Logical Volume Manager. Physical volumes combine to form the volume
group, which is then divided out in logical volumes to meet storage needs. (Images © 123RF.com.)
LVM provides a great deal of flexibility. Additional storage capacity can be added
to the VG and granted to the LV to provide storage space without interrupting or
reconfiguring user resources.
LVM maps whole physical devices and partitions (e.g., /dev/sda1, /dev/sdb2,
and so on) into one or more virtual structures called volume groups. Within these
volume groups are one or more logical volumes. Ultimately, the logical volumes
become the storage devices that the system, user, and applications work with.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The /dev/mapper/ directory contains all of the logical volumes on the system
that are managed by LVM. Devices in this directory are typically formatted as:
/dev/mapper/<volume group name>-<logical volume
name>
In some cases, this directory may just include links to the actual logical volume
location.
Many distributions support LVM, and several actually place the root file system on
logical volumes during installation.
Compared to traditional physical partition management, LVM provides the following
benefits:
• Dynamically create, delete, and resize volumes without having to reboot the
system.
• A logical volume can exceed the size of any one physical device (as long as it
doesn’t exceed the total size of devices in the volume group).
• Create virtual snapshots of each logical volume so you can quickly and easily
revert a volume to a specific state.
One potential downside to LVM is that the initial setup can be somewhat complex.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Storage Deployment
5
2. What role does the partprobe command play in the process of adding
storage?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 7C
Manage Other Storage Options
6
In some cases, direct-attached storage, such as internal drives, will not be sufficient
for the reliability, performance, or scalability requirements of a project. In such
cases, larger storage solutions, such as RAID, NAS, and SAN storage, may be
required. For security reasons, disk contents may need encryption.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Before encrypting a device, it’s a good idea to overwrite its contents with random data
or all zeros. This ensures that no sensitive data from past use remains on the device.
The shred command can be used to securely wipe a storage device in this manner.
The syntax is shred [options] {file-name}
RAID Standards
Three common RAID standards are RAID 0 (disk striping), RAID 1 (disk mirroring),
and RAID 5 (disk striping with parity).
Disk striping (RAID 0) relies on at least two disks. The disk partitions are divided
into sections called stripes and data is written sequentially through the stripes.
Because RAID 0 provides no fault tolerance, it is mainly used for performance
benefits with application data that does not need to be preserved (such as caching).
Disk mirroring (RAID 1) duplicates data on two storage disks. This provides
complete redundancy (100% of the data resides on each disk) but is a relatively
inefficient use of storage capacity. RAID 1 usually results in faster reads than a
single standalone disk.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Software RAID
Most servers contain hardware RAID controller cards. These cards provide a
performance benefit and allow flexible configurations. However, operating systems
such as Linux can configure and manage storage devices like RAID arrays. In Linux,
the mdadm command manages RAID. Think of the mdadm command as “multiple
device administration.”
To display information on a RAID array, use the mdadm command with a series of
flags. The flags set mdadm into different modes, such as Assemble, Monitor, and
Build.
The mdadm utility also provides a dynamic file that displays messages and status
information related to RAID arrays. To display the contents of this file, type:
cat /proc/mdstat
The output of the cat /proc/mdstat command displays the RAID level the
kernel currently supports and any current RAID configurations. The RAID levels are
referred to as “Personalities” and will exhibit values such as [raid1] or [raid5].
Any configured RAID arrays are identified on an md_d0 line, which consists of the
current RAID configuration and a list of disks that are members of the array. For
example, the output might look like the following:
Personalities : [raid0] [raid1] [raid5]
md_d0 : active raid1 sde1 sdf1
Network-Attached Storage
A network-attached storage (NAS) device is a dedicated network node that
provides inexpensive, scalable, and easy-to-configure storage. The device is
essentially an enclosure of storage drives managed by a minimal open-source
operating system.
Lesson 7: Administering Storage | Topic 7C
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The SMB protocol provides users with shared access to files and other resources
across a local area network (LAN). SMB clients make requests for resources
to SMB servers, which respond and provide the appropriate level of access. This
protocol is primarily used with Windows computers. However, SMB-compatible
software called Samba helps interface Linux and Windows hosts running network
shares.
Common Internet File System (CIFS) is a specific implementation of SMB.
Microsoft designed it as a successor to SMB version 1, but SMB versions 2 and 3
superseded it. However, Linux still uses the CIFS name in some of its tools, though
these tools support newer versions of SMB.
Network File System offers similar functionality to SMB, but the protocols are not
compatible. NFS is preferred in situations where Linux clients access Linux servers.
In environments that are a mix of Windows and Linux, the SMB protocol is the
better choice.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Storage-Area Network
A storage-area network (SAN) is a significantly more expensive and complex
option. However, it is usually faster and more scalable. Dedicated storage devices
are managed as a single unit and are controlled by one or more Linux servers.
Network protocols communicate between the management nodes and the storage
devices. Two common protocols are Fibre Channel (FC) and Internet Small
Computers Systems Interface (iSCSI).
In some cases, Fibre Channel devices may be attached to the server. The fcstat
command displays information about existing Fibre Channel adapters. These
adapters are typically found in conjunction with SAN solutions.
The syntax for the fcstat command is fcstat [options] {driver-
name}
For example, to display driver-collected information for the Fibre Channel
connection, type:
fcstat fcs0
The fcstat command also contains subcommands that provide additional detail,
such as link statistics. The syntax for displaying link statistics is:
fcstat link_stats {link-number}
A network segment featuring a SAN using Fibre Channel protocols. (Images © 123RF.com.)
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Identify multipathd
One common attribute of SAN implementation is multipathing. This concept refers
to providing multiple connections between a server and its SAN storage devices.
The benefit is fault tolerance (in the event one path becomes unavailable) and
performance (load balancing across paths). In Linux, multipathd is responsible
for checking and displaying information about paths.
The multipathd daemon manages the paths and reconfigures the network map as
needed to react to changes in paths (such as failures).
The multipathd command displays information about the state of paths.
For example, to display the current configuration, type multipathd show
config.
Network File Systems for SAN
Like standard file servers and NAS devices, SAN storage solutions may be
configured with NFS or SMB network filesystems. Users can connect to these shared
resources to access files and directories.
Note that the NFS and SMB protocols provide communication between network
clients and the filesystem. These operate on a higher level than iSCSI and Fibre
Channel, which govern the communication between the hosting server and
connected SAN storage.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Storage Options
7
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 7D
Troubleshoot Storage
4
Disk Performance
Storage devices read and write data, an action referred to as input/output (IO). Even
with solid-state storage, drive actions may be one of the slowest tasks the server
performs. Factor in many users reading and writing simultaneously, and it’s easy to
see why disk performance is a concern for sysadmins.
NVMe storage disk detection may fail during the initial installation of the drive or
during later boot sequences. Ensure the BIOS settings are correct for the storage
disk type and that the GRUB2 configuration file does not contain typos when
referencing the NVMe drive.
NVMe disks are also prone to failure from heat, so proper cooling is essential, and
frequent backups are recommended.
High Latency
Latency refers to the time it takes for data to travel across the communications
media. In the case of direct-attached storage (such as local storage disks), the media
is the motherboard, disk controllers, and connecting cables. Latency is any delay
as the data travels the path. Some latency is inevitable; the goal is to ensure the
latency is acceptable within the server’s performance requirements. The goal is for
this number to be as low as possible.
The organization’s individual needs define storage performance. Often these
needs will vary by service. For example, web services have different performance
requirements than database services. Hardware specifications, software, and
specific configurations impact disk IO performance and wait times. These variables
make it difficult to identify ways to decrease latency and improve throughput.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Low Throughput
Throughput is a measure of the storage device’s effective bandwidth, or ability
to move data to or from the drive. The goal is for this number to be as high as
possible.
Managing disk performance begins with purchasing a server with the appropriate
storage specifications. These specifications impact the drive’s throughput. Often,
this means SCSI or SAS storage, as SATA may not perform well enough for servers
with many storage devices. The drives themselves are important, too, not just the
control interface. For physical disks, the RPM value should be as high as possible—
often 15,000 RPMs.
Organizing storage into RAID arrays may also provide a performance benefit in
addition to fault tolerance. Hardware RAID controllers are typically preferred.
Measuring IOPS
You can check performance by keeping track of the Input/Output Operations
per Second (IOPS). Identifying throughput to ensure that disk performance is
acceptable on a Linux server involves the iostat and ioping commands.
The iostat utility generates reports on CPU and device usage. For storage, it
provides input and output statistics for block devices and partitions. Using the -d
option to specify device information only, the iostat command lists the following
statistics for each storage device:
• Transfers (I/O requests) per second (tps).
You can use this report to monitor how a storage drive is being used and to identify
any potential bottlenecks. For example, a faulty drive might have lower reads and/
or writes per second than expected. You can also use the report to help you decide
how to best distribute the I/O load between the available devices.
The syntax of the iostat command is iostat [options] [device
names]
The ioping command generates a report of device I/O latency in real time. It will
continuously “ping” the specified device with requests and print information about
each request at the command line. By default, this information tracks how long it
took an I/O request to finish. Aside from specifying a device to test, you can also
specify a path name to test whatever device is associated with that path.
Consider using ioping to troubleshoot latency issues with storage devices,
especially if you believe your read and/or write speeds are slower than they
should be.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Both iostat and ioping are useful for troubleshooting I/O errors that may
occur due to failing drives, performance issues, or storage disk driver problems.
Deadline
The Deadline scheduler performs sorting of I/O operations using three queues: a
standard pending request queue, a read first in first out (FIFO) queue, and a write
FIFO queue; the latter two are sorted by submission time and have expiration values.
When a request is submitted, it is sorted into the standard queue and placed at the end
of its appropriate FIFO queue. When the top request of the FIFO queue becomes older
than the queue’s expiration, the scheduler stops working with the standard queue and
starts servicing requests from the top of the FIFO queue—in other words, it switches to
the oldest requests. This ensures that the scheduler doesn’t “starve” a request for too
long. This makes it ideal for certain workloads like multi-threaded workloads.
CFQ
This refers to the Complete Fair Queuing (CFQ) scheduler. It is the default scheduler
for modern versions of the Linux kernel. In this scheduler, each process is given
its own queue, and each queue has an interval by which it is accessed (its time
slice). The scheduler uses a round-robin system to access each queue and services
requests from these queues until either their time slices or requests are exhausted.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
When the queue is empty, the CFQ waits 10 milliseconds for any new requests in
the queue, and if it doesn’t see any, it moves on to another queue. Like the deadline
scheduler, this helps to minimize request starvation. Its advantage is that it services
processes fairly and provides good performance in most workload situations.
Noop
Noop is the simplest scheduler and does not sort I/O requests, but merely
merges them. This can be ideal in situations where the device or its storage
controller performs its sorting operations. It can also benefit devices that don’t
have mechanical components requiring seek time, like SSDs and USB flash drives,
because this scheduler doesn’t expend much effort in reducing seek time. However,
the previous two schedulers are preferred in most other situations.
Set the scheduler on a particular device by modifying the scheduler file located at
/sys/block/<device name>/queue/scheduler.
Setting the scheduler is as simple as echoing the desired option to this file, as in:
echo noop > /sys/block/sda/queue/scheduler
Note that this sets the scheduler for runtime only; the setting will revert upon
reboot. To ensure your changes persist, you must modify the system’s boot loader
configuration.
Capacity Issues
One of the great challenges for sysadmins is managing storage capacity. Storage
space is relatively inexpensive, and there are many storage options, from direct-
attached storage to network storage to cloud storage. Users often see no need to
delete data, and many files are kept just in case they’re needed.
Display Storage
To manage storage, begin by understanding what storage devices exist. Tools such
as blkid and lsblk display information about recognized storage devices. Another
resource is the /proc/partitions file, which delineates the partitions that
Linux is aware of. Display this information with the cat /proc/partitions
command. Finally, both fdisk and parted display drive and partition
information.
If the storage devices are organized using LVM, commands such as pvdisplay,
vgdisplay, and lvdisplay provide information about the configured
volumes.
The mount command displays all attached storage and may be useful for
understanding exactly where particular storage devices reside (they won’t always
be local).
Find Issues
The du and df commands are commonly used to gather specific storage utilization
information. Both tools are flexible and have many options.
Use df to get general information on the mounted storage capacity. Use df with
a specific device path to gather more information about a given partition. For
example:
# df
# df /dev/sda2
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Inode Exhaustion
Each file created on a Linux partition is given an identifier by the filesystem. This
identifier—called an inode—uniquely identifies the file. Filesystems such as ext4
have a limited number of inodes available to allocate. While the available number is
very high, filesystems that store many small files may eventually run out of inodes
to assign. This is referred to as inode exhaustion.
If you receive an error message such as “No space left on drive” or “Cannot create
directory” but there is still storage capacity available on the drive, you’ve likely
encountered an inode exhaustion issue.
You can use the df and du commands again in this case to troubleshoot this issue.
The df -i command displays the percentage of inodes consumed and available
in the IUse% column of the output. If you have available storage space but are
receiving an error message when creating files, check this value.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Filesystem Issues
Filesystem corruption often occurs when Linux servers are not shut down
gracefully. This is more prevalent with non-journaled filesystems (most modern
filesystems, such as XFS and ext4, are journaled).
The fsck utility can be run on disks to detect and attempt to correct issues. Note
that the volume must be unmounted for fsck to be run against it. The fsck
command was discussed in Topic 7B.
Filesystem mismatch errors may be reported, especially after changes to the
filesystem or partition size. This issue may occur when an application (or the
system) attempts to write to the storage disk location where the filesystem
superblock resides. It may also happen if the storage disk itself is damaged or if
there is a misconfiguration of the filesystem or partition table.
You may be able to recover and access data by unmounting the filesystem (in
rescue mode) and using a repair utility such as fsck.
Device Issues
There are common troubleshooting steps for storage devices. Begin by ensuring
Linux is aware of the device. Commands such as blkid and lsblk display
storage device information. The hwinfo command can be filtered to show storage
disks, too. If the disk is not detected by Linux, confirm its physical connectivity
by checking the interface and power cable for a tight connection. If the drive is
installed in a motherboard slot, confirm the connection is good.
Once physical connectivity is established, Linux should be able to detect the drive.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
If issues with the filesystem are suspected, use the various ext4 or XFS tools to
confirm the status of the filesystem.
Verify the drive can be mounted by using the mount command. First, create
a directory that acts as a mount point, and then attach the storage capacity to
the directory. If the partition is not automatically mounted at boot, check for
typographical errors in /etc/fstab or the systemd.mount file.
Network file servers share directories by using NFS or SMB to make storage capacity
available from across the network. These remote filesystems may also be mounted.
When troubleshooting mount issues with remote filesystems, first ensure network
connectivity is established by using tools such as ping and ip addr. Next, verify
the network file server has made the directories available on the network and that
the user has permission to mount and read/write to the directories.
Network tools such as ping and ip addr are covered in a later Lesson.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Storage Troubleshooting
5
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Lesson 7
Summary
5
Proper administration of storage directly impacts the server’s performance and the
ability of users to access critical data. System specifications, such as SATA, SCSI, and
SAS play an important role, as do design decisions such as traditional partitioning
versus LVM-based storage. In fact, it may be more efficient to store data on NAS,
SAN, or cloud-based devices rather than on internal storage disks. Monitoring
performance and troubleshooting storage disk issues are common sysadmin tasks.
Guidelines
These best practices and guidelines are provided for your use as revision tools or as
quick references in your job role.
• Understand the process of configuring storage with traditional partitions.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
LESSON INTRODUCTION
Linux sysadmins must identify and configure hardware resources. First, administrators
must be able to find hardware information using a series of commands to identify CPU,
memory, bus, and other hardware devices. Next, sysadmins manipulate processes
consuming hardware resources to optimize and troubleshoot the system. Often the
focus of this optimization is the CPU and memory. Finally, the Linux kernel—the core
of the operating system—may be managed to maintain performance, enable modern
features, and provide compatibility with current hardware.
Lesson Objectives
In this lesson, you will:
• Gather hardware information.
• Manage processes.
• Manage memory.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 8A
Gather Hardware Information
2
• USB controllers
• Graphics cards
• Print devices
One common option used with hwinfo is --short, which returns abbreviated
information on the resources. Using either hwinfo or hwinfo --short
displays information on all detected hardware. However, it’s more likely you’ll want
details for specific devices.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
The output from dmidecode may be very long. Use the pipe redirector to display
the information in a pagination utility such as less or more.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Recall that in the previous section, you displayed storage device information from the
/dev/sda file.
It’s possible to test the udev device manager by using the monitor subcommand.
Run udevadm monitor and then attach a new USB device to the system. You
should see the device is detected and the output shows device details.
Information can be displayed for specific attached devices by using the udevadm
command. For example, type udevadm info /dev/sda to see storage
device information.
The term “storage device” represents traditional magnetic hard disk drives, modern
solid-state drives, USB flash drives, and any other media used to store data.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Three other device files are listed: null, zero, and urandom. These are known
as the special device files.
The /dev/null special file is a writeable location that is used as a target for
generated data that should be discarded. This data may be error messages that are
redirected to /dev/null by using the 2> redirector.
The /dev/zero file is a way of filling storage capacity. For example, using the
dd command, a sysadmin can create a file of a specified size as part of testing. For
example, to create a one MiB file named foo.txt, run the following command:
# dd if=/dev/zero of=foo.txt count=1024 bs=1024
This command uses /dev/zero as the source of data that fills the file to its
specified size.
The /dev/urandom file can create is a source of random characters for
tasks such as creating completely randomized passwords. Take care of how this
command runs on low-performing systems at the risk of consuming all available
resources.
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
This section covers installed memory, not how that memory is being utilized. Memory
utilization is covered in a future section.
Memory Tools
Use the lsmem command to display memory blocks, including their size and state
(online, offline).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
File contents might be useful when troubleshooting memory. For example, if you
believe the system has 64 GB of RAM installed, but only 32 GB are reported, you
can use this file to see what memory Linux recognizes. Doing so may help identify
which sticks of RAM are not recognized (and therefore, may not be seated in the
motherboard correctly).
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Review Activity:
Hardware Information
3
4. What file has the make, model, number of cores, and other details
about the CPU?
LICENSED FOR USE ONLY BY: PRANAV KAMBLE · 31662591 · OCT 10 2022
Topic 8B
Manage Processes
5
Processes are instances of running code. These may be parts of the OS, services,
applications, or scripts. Some processes are executed as part of the system, while
others are initiated and managed by users. Each process is assigned a unique
process identification number (called a PID), and this number is used to manage
the process throughout its life cycle. This section covers process management,
troubleshooting, and optimization.
Processes Concepts
Process management is an essential part of performance monitoring and
troubleshooting on Linux systems. Sysadmins manage processes with tools that
indicate their resource consumption, status, and priority levels.