Linux Pocket Guide, 4th Edition

Chapter 1. Essential Concepts

What’s Linux?

Linux is a free, open source operating system (OS) that’s an alternative to Microsoft Windows and Apple macOS. Linux powers most of the servers on the internet. It operates behind the scenes on every Android mobile phone and Chromebook, and on millions of network-connected devices like routers, firewalls, and robotic cow-milking systems (seriously). It also runs fine on desktop and laptop computers.

Linux has four major parts, shown in Figure 1-1:

The kernel

Low-level software. It controls the hardware and basic functions like process scheduling and networking. Few users interact with the kernel directly.

Supplied programs

Thousands of programs for file handling, text editing, software development, web browsing, audio, video, encryption, mathematics…you name it. These programs talk to the kernel. Programs that run on the command line are called commands.

The shell

A Linux program for running commands and displaying the results. Linux has an assortment of shells with different features. This book focuses on a shell called bash, which is often the default for user accounts. Some other shells are dash, fish, ksh (Korn shell), tcsh (TC shell, or T shell), zsh (Z shell), and to a lesser extent, busybox. All shells have similar functions, though their usage varies.

Graphical desktop environment (optional)

A UI with windows, menus, icons, mouse support, and other familiar GUI elements. Some popular environments are GNOME and KDE Plasma. Most applications built for GNOME can run in KDE and vice versa.¹

This book focuses on the command-line parts of Linux, namely the supplied programs and the shell. Windows and macOS have command-line interfaces too (cmd and powershell on Windows, Terminal on the Mac), but most of their users stick with the GUI and might never see or need a command line. On Linux, the shell is critical. If you use Linux without the shell, you are missing out.

Linux is extremely configurable and comes in hundreds of varieties that serve different needs and tastes. Each variety is called a distro (short for “distribution”). All distros share some core components but may look different and include different programs and files. Some popular distros include Mint, Ubuntu, Manjaro, Arch, Gentoo, Red Hat, and OpenSUSE, among others. The core material in this book should apply to every distro.

Figure 1-1. The four major parts of Linux, conceptually. Low-level kernel functions are called by programs, which are invoked in a shell, which can be launched by a graphical desktop.

Launching a Shell

Where do shells come from? Sometimes Linux will launch one for you automatically. This is often the case when you log in over a network using ssh or a similar tool. The first significant thing you see is a shell prompt awaiting your command.

Other times, you have to launch a shell manually. This is common when using a graphical desktop full of icons and menus with no shell in sight. In such cases, you need a GUI application called a terminal or terminal program that runs shells in a window. The sidebar “Shell Versus Terminal” clarifies the difference between shells and terminals.

Every distro with a graphical desktop includes at least one terminal program, but you might have to hunt for it. Search for an application, icon, or menu item named Terminal, Konsole, xterm, gnome-terminal, uxterm, or something similar, and run it to open a terminal. Also try pressing Ctrl-Alt-t (hold the Control and Alt keys and press T), which opens a terminal in some environments.

Shell Versus Terminal

A shell is a command-line interface for launching Linux commands by typing plain text. It prints a prompt and waits for your command:

→

A terminal is a program that opens a window and presents a running shell, shown in Figure 1-2. It’s like a graphical wrapper around a shell. A terminal adds menus, scrollbars, copy and paste, and other GUI features that support the shell.

Figure 1-2. A terminal program opens a window that runs a shell.

Command-Line Warm-Up

To give you a feel for Linux, here are 10 simple commands to try right now in a shell. Type them exactly, including capital and small letters, spaces, and all symbols after the prompt. At the end of each command, press Enter.²

Display a calendar for November 2023:

→ cal nov 2023
   November 2023
Su Mo Tu We Th Fr Sa
          1  2  3  4
 5  6  7  8  9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30

List the contents of the /bin directory, which contains many commands:

→ ls /bin
bash      less        rm
bunzip2   lessecho    rmdir
busybox   lessfile    rnano
⋮

Count the number of visible items in your home directory (represented here by a variable, HOME, that I discuss later):

→ ls $HOME | wc -l
8                     Your value may be different

See how much space is used on a partition of your hard disk:

→ df -h /
Filesystem  Size  Used Avail Use% Mounted on
/dev/sdb1    78G   30G   48G  61% /

Watch the processes running on your computer (press “q” to quit):

→ top -d1

Print the file /etc/hosts, which contains names and addresses of computers, on your default printer if you have one set up:

→ lpr /etc/hosts

See how long you’ve been logged in:

→ last -1 $USER
smith  pts/7 :0  Tue Nov 10 20:12  still logged in

Download a file sample.pdf from this book’s website to your current directory, without needing a web browser:

→ curl -O https://linuxpocketguide.com/sample.pdf

See who owns the domain name oreilly.com (press the space bar to move forward page by page, and press “q” to quit):

→ whois oreilly.com | less
Domain Name: OREILLY.COM
Registrar: GODADDY.COM, LLC
⋮

Finally, clear your terminal or screen:

→ clear

Congratulations, you are now a Linux user!

The Structure of Commands

A Linux command typically consists of a program name followed by options and arguments:

wc -l myfile

The program name (wc, short for “word count”) refers to a program somewhere on disk that the shell locates and runs. Options, which usually begin with a dash, affect the behavior of the program. In the preceding command, the -l option tells wc to count lines and not words. The argument myfile specifies the file that wc should read and process.

Commands can have multiple options and arguments. Options may be given individually, or combined after a single dash:

wc -l -w myfile           Two individual options
wc -lw myfile             Combined options, same as -l -w

though some programs are quirky and do not recognize combined options. Multiple arguments are also OK:

wc -l myfile myfile2      Count lines in two files

Options are not standardized. They may be a single dash and one character (say, -l), two dashes and a word (--lines), or several other formats. The same option may have different meanings to different programs: in the command wc -l, the option -l means “lines of text,” but in ls -l it means “long output.” Two programs also might use different options to mean the same thing, such as -q for “run quietly” versus -s for “run silently.” Some options are followed by a value, such as -s 10, and space between them might not be required (-s10).

Arguments are usually filenames for input or output, but they can be other things too, like directory names, usernames, hostnames, IP addresses, regular expressions, or arbitrary strings.

A command that’s just a single program with options and arguments is called a simple command. Here’s a simple command that lists users who are logged into a Linux server:³

→ who
silver       :0    Sep 23 20:44
byrnes    pts/0    Sep 15 13:51
barrett   pts/1    Sep 22 21:15
silver    pts/2    Sep 22 21:18

A command can also invoke several programs at once and even connect programs so they interact. Here’s a command that connects the output of who to the input of wc, which counts lines of text. The result is the number of lines in the output of who:

→ who | wc -l
4

The vertical bar, called a pipe, makes the connection between who and wc. Linux experts use these sorts of combined commands, called pipelines, all the time.

Commands can also include programming language constructs like variables, conditionals, and loops, which I cover in “Programming with Shell Scripts”. For example, a command might say, “run this program, write its output to a file of my choosing, and if any errors occur, send me an email with the results.”

Users and Superusers

Linux is a multiuser OS: multiple people can run programs on a single Linux computer at the same time. On a given computer, each user is identified by a username, like smith or funkydance. Each user has a separate workspace of sorts (see “Home Directories”) so they don’t interfere with one another.

A special user named root—the superuser or administrator—has the privileges to do anything at all on the system. The superuser can create, modify, or delete any file and run any program. Ordinary users are restricted: they can run most programs, but in general, they can’t mess with other users’ stuff.

Some commands in this book require superuser privileges. I precede these commands with sudo:

→ sudo superuser command goes here

Warning

sudo gives you the power to destroy your Linux system.

I discuss sudo fully in “Becoming the Superuser”, but for now, all you need to know is that sudo gives you superuser powers and sometimes prompts for your password. For example, to count lines in a protected file called /etc/shadow, with and without sudo, you could run this command:

→ wc -l /etc/shadow                 This fails
wc: /etc/shadow: Permission denied
→ sudo wc -l /etc/shadow            Run with sudo
[sudo] password: xxxxxxxx
51 /etc/shadow                      It worked!

The Filesystem

To make use of any Linux system, you must become comfortable with Linux files and directories (a.k.a. folders), collectively called the filesystem. On a graphical desktop, files and directories are obvious on screen. In a command-line interface like the Linux shell, the same files and directories are still present but less visible, so at times you must remember which directory you are “in” and how it relates to other directories. You’ll use shell commands like cd (change directory) to move between directories, and commands like pwd (print working directory) to keep track of where you are in the filesystem.

Let’s cover some terminology. Linux files are collected into directories. The directories form a hierarchy, or tree, as in Figure 1-3. One directory may contain other directories, called subdirectories, which may themselves contain other files and subdirectories, and so on, into infinity. The topmost directory is called the root directory and is denoted by a slash (/).⁴

Figure 1-3. A Linux filesystem (partial). The root directory is at the top. The absolute path to the “dan” directory is /home/dan.

Linux refers to its files and directories using a “names and slashes” syntax called a path. For instance, the following path:

/one/two/three/four

refers to the root directory /, which contains a directory called one, which contains a directory two, which contains a directory three, which contains a final file or directory, four. Any path that begins with a slash, which descends all the way from the root, is called an absolute path.

Paths don’t have to be absolute—they can be relative to some directory other than the root. Figure 1-3 has two different directories named bin, whose absolute paths are /bin and /usr/bin. If you simply refer to “the bin directory,” it’s not clear which one you mean. You need more context. Any path that doesn’t begin with a slash, like bin, is called a relative path.

To make sense of a relative path, you need to know “where you are” in the Linux filesystem. This location is called your current directory (sometimes called “working directory” or “current working directory”).

Every shell has a current directory, and when you run commands in that shell, they operate relative to its current directory. For example, if your shell is “in” the directory /usr, and you refer to a relative path bin, it means /usr/bin. In general, if your current directory is /one/two/three, a relative path a/b/c would imply the absolute path /one/two/three/a/b/c.

Two special relative paths are named . (a single period, or “dot”) and .. (two periods, or “dot dot”). “Dot” means your shell’s current directory, and “dot dot” means its parent directory, one level above. So if your shell’s current directory is /one/two/three, then . refers to this directory and .. refers to /one/two.

To travel from one directory to another, use the cd command, which changes your shell’s current directory:

→ cd /usr/local/bin       Enter the directory /usr/local/bin

The previous cd command used an absolute path. You can make relative moves with cd as well:

→ cd d           Enter subdirectory d of my current directory
→ cd ../mydir    Go up to my parent, then into directory mydir

File and directory names may contain most characters you expect: capital and small letters,⁵ numbers, periods, dashes, underscores, and most symbols (but not “/”, which is reserved for separating directories). For efficiency, however, avoid spaces, asterisks, dollar signs, parentheses, and other characters that have special meaning to the shell. These characters require special treatment in filenames (see “Quoting”), which can be inconvenient or tedious.

Home Directories

Users’ personal files are usually kept in the directory /home (for ordinary users) or /root (for the superuser). Your home directory is typically /home/<your-username> (/home/smith, /home/funkydance, etc.). There are several ways to visit or refer to your home directory:

cd

With no arguments, the cd command returns you (i.e., sets the shell’s current directory) to your home directory.

HOME variable

The environment variable HOME (see “Shell Variables”) contains the name of your home directory:

→ echo $HOME                 Print the directory name
/home/smith
→ cd $HOME/linuxpocketguide  Visit a subdirectory

~

When used in place of a directory, a lone tilde is expanded by the shell to the name of your home directory.

→ echo ~                  Print the directory name
/home/smith
→ cd ~/linuxpocketguide   Visit a subdirectory

When the tilde is followed by a username (as in ~fred), the shell expands this string to be the user’s home directory:

→ cd ~fred       Visit Fred's home directory, if it exists
→ pwd            The “print working directory” command
/home/fred

System Directories

A Linux system has tens of thousands of system directories. They contain OS files, applications, documentation, and just about everything except personal user files (which typically live in /home).

Unless you’re a system administrator, you’ll rarely visit most system directories—but with a little knowledge you can understand or guess their purposes. Their names often contain three parts, as shown in Figure 1-4.

Figure 1-4. Directory scope, category, and application

Directory path part 1: scope

The scope of a directory path describes, at a high level, the purpose of an entire directory tree. Some common ones are:

/	(Pronounced “root”) System files supplied with your distro
/usr	(Pronounced “user”) More system files supplied with your distro
/usr/local	(Pronounced “user local”) System files that are not supplied with your distro; they may be unique to your local Linux network or your individual computer

There isn’t a clear distinction between / and /usr in practice, but / is considered “lower level” and closer to the OS.

Directory path part 2: category

The category of a directory path in Figure 1-4 describes the types of files found in a directory. For example, if the category is lib, you can be reasonably sure that the directory contains library files for programming. If the category is bin, the contents are usually binary files—executable programs.

When you precede a category like bin with a scope, you produce paths like /bin, /usr/bin, and /usr/local/bin. A distro’s most fundamental system programs like ls and cat are typically in /bin, and other system programs are in /usr/bin.⁶ /usr/local/bin contains locally installed programs not included in your distro. These are not hard-and-fast rules but typical cases.

Some common categories are as follows:

Categories for programs
bin	Programs (usually binary files)
sbin	Programs (usually binary files) for superusers
lib	Libraries of code used by programs
Categories for documentation
doc	Documentation
info	Documentation files for emacs’s built-in help system
man	Documentation files (manual pages) displayed by the man command; the files are often compressed and are sprinkled with typesetting commands for man to interpret
share	Program-specific files, such as examples and installation instructions
Categories for configuration
etc	Configuration files for the system (and other miscellaneous stuff)
init.d	Configuration files for booting Linux
rc.d	Configuration files for booting Linux; also rc1.d, rc2.d, …​
Categories for programming
include	Header files for programming
src	Source code for programs
Categories for web files
cgi-bin	Scripts/programs that run on web pages
html	Web pages
public_html	Web pages, typically in users’ home directories
www	Web pages
Categories for display
fonts	Fonts (surprise!)
X11	X window system files
Categories for hardware
dev	Device files for interfacing with disks and other hardware
media	Mount points: directories that provide access to disks
mnt	Mount points: directories that provide access to disks
Categories for runtime files
var	Files related to the state of the computer, updated frequently
lock	Lock files, created by programs to say, “I am running”; the existence of a lock file may prevent another program, or another instance of the same program, from performing an action
log	Logfiles that track important system events, containing error, warning, and informational messages
mail	Mailboxes for incoming email
run	PID files, which contain the IDs of running processes; these files are often consulted to track or kill particular processes
spool	Files queued or in transit, such as outgoing email, print jobs, and scheduled jobs
tmp	Temporary storage for programs and/or people to use

Directory path part 3: application

The application part of a directory path (Figure 1-4), if present, is usually the name of a program. For example, the directory /etc/systemd has scope root (/), category etc (configuration files), and application systemd. Since systemd is a service for configuring Linux machines, a good guess is that /etc/systemd contains configuration files for that service—and it does.

Kernel-Related Directories

Some directories support the Linux kernel, the lowest-level part of the Linux OS:

/boot

Files for booting the system. The kernel lives here, typically in /boot/vmlinuz or a file of similar name.

/lost+found

Damaged files that were rescued by a disk recovery tool.

/proc

Files for currently running processes; for advanced users.

/sys

Files for kernel internals; for advanced users.

The files in /proc and /sys provide views into the running kernel and have special properties. Files in /proc always appear to be zero-sized, read-only, and dated now, but their contents magically contain information about the Linux kernel:

→ ls -lG /proc/version
-r--r--r--  1 root  0 Oct  3 22:55 /proc/version
→ cat /proc/version
Linux version 5.15.0-76-generic ...

Files in /sys also have misleading sizes and magical contents:

→ ls -lG /sys/power/state
-rw-r--r-- 1 root 4096 Jul  8 06:12 /sys/power/state
→ cat /sys/power/state
freeze mem disk

/proc and /sys are used mostly by system programs, but feel free to view them. Here are some examples:

/proc/ioports	A list of your computer’s input/output hardware.
/proc/cpuinfo	Information about your computer’s processors.
/proc/version	The OS version. The uname command prints the same information.
/proc/uptime	System uptime: seconds elapsed since the system was last booted. Run uptime for a more human-readable result.
/proc/NNN	Information about the Linux process with ID NNN, where NNN is a positive integer, such as /proc/13542.
/proc/self	Information about the current process you’re running; a symbolic link to a /proc/nnn file, automatically updated. Try running: → ls -l /proc/self several times in a row, and /proc/self changes where it points.

File Permissions

A Linux system may have many user accounts. To maintain privacy and security, most users can access only some files on the system, not all. This access control is embodied in two questions:

Who has permission?

Every file and directory has an owner who can do anything they want with it. Typically, a file’s owner is the user who created it. A superuser can change a file’s owner.

Additionally, a predefined group of users may access a file. Groups are defined by the system administrator and I cover them in “Group Management”.

Finally, a file or directory can be opened to all users who have accounts on the system. You’ll also see this set of users called the world or simply other.

What kind of permission is granted?

File owners, groups, and the world may each have permission to read, write (modify), and execute (run) particular files. Permissions also extend to directories, which users may read (view files within the directory), write (create and delete files within the directory), and execute (enter the directory with cd).

To see the ownership and permissions of a file named myfile, run ls -l, described in “Basic File Operations”:

→ ls -l myfile
-rw-r--r-- 1 smith smith  1168 Oct 28  2015 myfile

To see the ownership and permissions of a directory named mydir, add the -d option:

→ ls -ld mydir
drwxr-x--- 3 smith smith  4096 Jan 08 15:02 mydir

In the output, the file permissions are the 10 leftmost characters, a string of r (read), w (write), x (execute), other letters, and dashes. For example:

-rwxr-x---

Here’s what these letters and symbols mean, briefly:

Position	Meaning
1	File type: - = file, d = directory, l = symbolic link, p = named pipe, c = character device, b = block device
2–4	Permissions for the file’s owner: read (r), write (w), execute (x), or no permission (-).
5–7	Permissions for the file’s group: r, w, x, -
8–10	Permissions for all other users: r, w, x, -

My example -rwxr-x--- means a file that can be read, written, and executed by the owner; read and executed by the group, but not written; and not accessed at all by other users. To change the owner, group, or permissions, use the commands chown, chgrp, and chmod, respectively, as described in “Properties of Files”.

Selected Features of Bash

A shell does much more than simply run commands. It also simplifies the running of commands, thanks to powerful features: pattern matching for filenames, a “command history” to recall previous commands quickly, pipes to send the output of one command to the input of another, variables to store values for use by the shell, and more. Take the time to learn these features, and you will become faster and more productive with Linux.⁷ Let’s skim the surface and introduce you to these useful tools. (For full documentation, run info bash.)

Which Shell Are You Running?

This book assumes that your shell is bash. To identify your shell, run:

→ echo $SHELL
/bin/bash

If your shell isn’t bash and you wish to try it, run the command bash directly, because bash, like all shells, is just a program. (It’s located at /bin/bash.)

→ bash

Run the command exit when done to return to your regular shell. To change your default shell to bash, see the chsh command in “User Account Management”.

Pattern Matching

Pattern matching in the shell, sometimes called wildcards, is a shorthand to work with sets of files. For example, the pattern a* refers to files whose names begin with lowercase “a.” The shell expands a pattern into the full set of filenames it matches. If you run:

→ ls a* 
aardvark  adamantium  apple

the shell invisibly expands the pattern a* into the filenames that begin with “a” in your current directory, as if you had typed:

→ ls aardvark adamantium apple

ls never knows you used a pattern: it sees only the final list of filenames after expansion. This means every Linux program you launch from a shell, regardless of its origin, “works” with patterns and other shell features. This is a critically important point. A surprising number of Linux users think that programs expand their own file patterns on the command line. They don’t. The shell does it before the associated program even runs.

Patterns never match two special characters: a leading period and the directory slash (/). These characters must be given literally. A pattern like .bas* matches .bashrc, and /etc/*conf matches all filenames ending in conf in the /etc directory.

Dot Files

Filenames with a leading period, called dot files, are often hidden from view unless you explicitly request them. An example is the bash initialization file .bashrc in your home directory.

• ls omits dot files from directory listings, unless you provide the -a option.

• Pattern-matching characters in the shell do not match a leading period.

As a result, dot files are often called “hidden files.”

Pattern	Meaning
*	Zero or more consecutive characters, except a leading dot or a directory slash.
?	Any single character, except a leading dot or a directory slash.
[set]	Any single character in the given set. It can be a sequence of characters, like [aeiouAEIOU] for all vowels, or a range with a dash, like [A-Z] for all capital letters, or a combination.
[!set]	Any single character not in the given set, such as [!0-9] to mean any nondigit.
[^set]	Same as [!set].

To match a literal dash in a character set, put it first or last so it’s not part of a range. To include a literal closing square bracket in the set, put it first in the set, or escape it with a backslash (\]). To include a ^ or ! symbol literally, place it somewhere other than first in the set, or escape it.

Brace Expansion

Similar to file patterns, expressions with curly braces also expand to become multiple arguments to a command. The comma-separated expression:

{bubble,quick,merge}

expands first to bubble, then quick, and finally merge within a command line, like this:

→ echo {bubble,quick,merge}sort.java
bubblesort.java quicksort.java mergesort.java

Note

The key difference between braces and square brackets is that braces work with any strings, whereas square bracket expressions match only existing filenames.

Curly braces can also expand to a sequence of values in a range, if you separate the endpoints of the range with two dots (..):

→ echo {3..12}
3 4 5 6 7 8 9 10 11 12
→ echo {A..E}
A B C D E
→ echo file{1..5}.py
file1.py file2.py file3.py file4.py file5.py

Shell Variables

You can define variables in a shell and assign them values:

→ MYVAR=3      Assign the value 3 to variable MYVAR

To refer to a value, simply place a dollar sign in front of the variable name:

→ echo $MYVAR
3

The shell defines some standard variables when you log in:

Variable	Meaning
DISPLAY	The name of your X window display
HOME	Your home directory, such as /home/smith
LOGNAME	Your login name, such as smith
MAIL	Your incoming mailbox, such as /var/spool/mail/smith
OLDPWD	Your shell’s previous directory, prior to the last cd command
PATH	Your shell search path: directories separated by colons
PWD	Your shell’s current directory
SHELL	The path to your shell (e.g., /bin/bash)
TERM	The type of your terminal (e.g., xterm or vt100)
USER	Your login name

Variables and their values are limited, by default, to the shell that defines them. To make a variable and its value available to other programs your shell invokes (i.e., subprocesses), use the export command:

→ MYVAR=3
→ export MYVAR

or the shorthand:

→ export MYVAR=3

Your exported variable is now called an environment variable. To go further and make a variable available to every new shell you run, not just subprocesses of your current shell, place the variable definition beforehand in a shell configuration file; see “Tailoring Shell Behavior”.

To list a shell’s environment variables, run:

→ printenv

To set an environment variable just for the duration of one command, prepend variable=value to the command line:

→ printenv HOME
/home/smith
→ HOME=/home/sally printenv HOME
/home/sally
→ printenv HOME
/home/smith            The original value is unaffected

Search Path

Programs may be scattered all over the Linux filesystem, mostly in directories like /bin and /usr/bin. When you run a command that invokes a program, somehow the shell must locate the program in the filesystem:

→ who         The shell must locate the “who” program to run it

The shell finds the program by consulting the value of the environment variable PATH, which is a list of directories separated by colons. This list is called the shell’s search path.

→ echo $PATH
/usr/local/bin:/bin:/usr/bin    Search path with 3 directories

The shell looks for an executable file named who in each listed directory in sequence. If it locates who (say, in /usr/bin/who), it executes the program, and also caches the location for next time (run hash --help for more on caching). Otherwise, it reports a failure:

bash: who: command not found

To print a command’s location in your search path, run the type or which command:

→ type who                     The output may vary:
who is hashed (/usr/bin/who)   This means "who" is cached
who is /usr/bin/who            This means "who" isn't cached
→ which who
/usr/bin/who

To add directories to your shell’s search path temporarily, modify its PATH variable. For example, append /usr/sbin to your shell’s search path:

→ PATH=$PATH:/usr/sbin
→ echo $PATH
/usr/local/bin:/bin:/usr/bin:/usr/sbin

This change affects only the current shell. To make it stick, modify PATH in a bash configuration file, as explained in “Tailoring Shell Behavior”. Then log out and log back in, or run the configuration file by hand in each open shell window. For example:

→ . $HOME/.bashrc    If you modified $HOME/.bashrc

Aliases

The alias command defines a convenient shorthand for another command. For example, this alias:

→ alias ll='ls -lG'

defines a new command ll that runs ls -lG:

→ ll
total 436
-rw-r--r--    1 smith     3584 Oct 11 14:59 file1
-rwxr-xr-x    1 smith       72 Aug  6 23:04 file2
⋮

Define aliases in your ~/.bashrc file (see “Tailoring Shell Behavior”) to make them available to future shells.⁸ To list all your aliases, run alias. If you want more flexibility than aliases provide, see “Programming with Shell Scripts”, run info bash, and read up on “shell functions.”

Built-in Commands

Most Linux commands are programs in the Linux filesystem. Examples are wc and who, which usually live in the directory /usr/bin. The shell locates and runs them using the PATH variable, as I described in “Search Path”. Some other commands, however, are built-in features of the shell, known as built-in commands. You’ve seen several built-in commands in this chapter, such as cd, alias, and export. To determine whether a command is in the filesystem, a built-in command, or an alias, run the type command:

→ type wc cd ll             Print the types of these commands
wc is /usr/bin/wc           A program in the filesystem
cd is a shell builtin       A built-in shell command
ll is aliased to `ls -lG'   An alias

Input, Output, and Redirection

Most Linux commands accept input and/or produce output. Keyboard input is called standard input or stdin. Output to your display is called standard output or stdout. Error messages are treated specially and printed on standard error or stderr, which also is usually your display, but Linux separates stderr from stdout internally.⁹

The shell can redirect standard input, standard output, and standard error to and from files. In other words, any command that reads from standard input can have its input come from a file instead with the shell’s < operator:

→ command < infile

Likewise, any command that writes to standard output can write to a file instead:

→ command > outfile      Create/overwrite outfile
→ command >> outfile     Append to outfile

A command that writes to standard error can have its error output redirected to a file as well, leaving standard output unaffected:

→ command 2> errorfile

To redirect both standard output and standard error to files:

→ command > outfile 2> errorfile  Separate files
→ command &> outfile              Single file (preferred)
→ command >& outfile              Single file (less common)

Combined Commands

Bash lets you move beyond simple commands by combining multiple programs on a single command line.

Sequences of commands

To invoke several commands in sequence on a single command line, separate them with semicolons:

→ command1 ; command2 ; command3

To run a sequence of commands as before, but stop execution if any of them fails, separate them with && (“and”) symbols:

→ command1 && command2 && command3

To run a sequence of commands, stopping execution as soon as one succeeds, separate them with || (“or”) symbols:

→ command1 || command2 || command3

Pipes

You can redirect the standard output of one command to be the standard input of another, using the shell’s pipe (|) operator. (On US keyboards, find this symbol just above the Enter key.) For example, this command:

→ who | sort

sends the output of who into the sort command, printing an alphabetically sorted list of logged-in users. Multiple pipes work too. Let’s sort the output of who again, extract the first column of information (using awk), and display the results one page at a time (using less):

→ who | sort | awk '{print $1}' | less

Command substitution

If you surround a command with backquotes (“backticks”), the shell removes the command and substitutes the command’s output.

→ date +%Y                 Print the current year
2024
→ echo This year is `date +%Y`
This year is 2024

A dollar sign and parentheses are equivalent to backquotes:

→ echo This year is $(date +%Y)
This year is 2024

but are superior because they can be nested:

→ echo Next year is $(expr $(date +%Y) + 1)
Next year is 2025

Process substitution

Some programs don’t work well with pipes because they don’t read from standard input, only from disk files. An example is the diff command that compares two files line by line and prints their differences. Process substitution is a way to force a command like diff to read from standard input. It runs a command and lets its output “masquerade” as a file, which programs like diff will happily accept. With the process substitution operator, <(), you can compare the output of two commands instead of two disk files.

Suppose you have a directory full of JPEG and text files in pairs:

→ ls jpegexample
file1.jpg  file2.jpg  file3.jpg  ...
file1.txt  file2.txt  file3.txt  ...

and you want to confirm that every JPEG file has a corresponding text file and vice versa. Ordinarily, you might create two temporary files, one containing the JPEG filenames and the other containing the text filenames, remove the file extensions with cut, and compare the two temporary files with diff:

→ cd jpegexample
→ ls *.jpg | cut -d. -f1 > /tmp/jpegs
→ ls *.txt | cut -d. -f1 > /tmp/texts
→ diff /tmp/jpegs /tmp/texts
5a6
> file6       No file6.jpg was found
8d8
< file9       No file9.txt was found

Process substitution performs the same task with a single command and no temporary files:

→ diff <(ls *.jpg|cut -d. -f1) <(ls *.txt|cut -d. -f1)

Each <() operator stands in for a filename on the command line, as if that “file” contained the output of ls and cut.

Preventing Evaluation

The shell evaluates every character of a command. To prevent evaluation, use quoting or escaping.

Quoting

Normally, the shell treats whitespace as a separator for strings on the command line. To make a string that contains whitespace (e.g., a filename with a space in it), surround it with single or double quotes, and the shell treats it as a unit. Single quotes treat their contents literally, while double quotes permit variables and other shell constructs to be evaluated:

→ echo 'The variable HOME has value $HOME'
The variable HOME has value $HOME
→ echo "The variable HOME has value $HOME"
The variable HOME has value /home/smith

Escaping

If a character has special meaning to the shell but you want it used literally (e.g., * as a literal asterisk rather than a file pattern), precede the character with the backslash “\” character. This is called escaping the special character:

→ echo a*                    A file pattern
aardvark  adamantium  apple
→ echo a\*                   A literal asterisk
a*
→ echo "I live in $HOME"     Print a variable value
I live in /home/smith
→ echo "I live in \$HOME"    Print a literal dollar sign
I live in $HOME

You can also escape control characters (tabs, newlines, ^D, etc.) to have them used literally on the command line, if you precede them with ^V. This is particularly useful for tab characters, which the shell would otherwise use for filename completion (see “Filename Completion”).

→ echo "There is a tab between here^V    and here"
There is a tab between here      and here

Command-line Editing

Bash lets you edit the command line you’re working on, using keystrokes inspired by the text editors Emacs and Vim (see “Creating and Editing Files”). To enable command-line editing with Emacs keys, run this command (and place it in a bash configuration file to make it permanent):

→ set -o emacs

For vi (or Vim) keys:

→ set -o vi

Emacs keystroke	Vim keystroke (after ESC)	Meaning
^P or up arrow	k or up arrow	Go to previous command
^N or down arrow	j or down arrow	Go to next command
^R		Search for a previous command interactively
^F or right arrow	l or right arrow	Go forward one character
^B or left arrow	h or left arrow	Go backward one character
^A	0	Go to beginning of line
^E	$	Go to end of line
^D	x	Delete next character
^U	^U	Delete to beginning of line

Command History

A shell can recall previous commands and re-execute them, a feature called command history. Try these useful history-related commands and expressions:

Command	Meaning
history	Print your history
history N	Print the most recent N commands in your history
history -c	Clear (delete) your history
!!	Represents your previous command. To re-run it: → !! <Enter>
!N	Represents command number N in your history
!-N	Represents the command you entered N commands ago
!$	Represents the last argument from the previous command. Great for checking that files are present before running a destructive command like rm: → ls z* zebra.txt zipfile.zip zookeeper → rm !$ Same as “rm z*”
!*	Represents all arguments from the previous command: → ls myfile emptyfile hugefile emptyfile hugefile myfile → wc !* 18 211 1168 myfile 0 0 0 emptyfile 333563 2737540 18577839 hugefile 333581 2737751 18579007 total

Filename Completion

In the middle of typing a filename, press the Tab key and the shell automatically completes the filename for you. If several filenames match what you’ve entered so far, the shell beeps, indicating the match is ambiguous. Immediately press Tab a second time and the shell presents the alternatives. Try this:

→ cd /usr/bin
→ ls un<Tab><Tab>

The shell displays all files in /usr/bin that begin with un, such as uniq and unzip. Enter a few more characters to disambiguate your choice and press Tab again.

Shell Job Control

jobs	List your jobs.
&	Placed after a command, runs it in the background.
^Z	Keystroke to suspend the current (foreground) job.
suspend	Suspend a shell.
fg	Unsuspend a job: bring it into the foreground.
bg	Make a suspended job run in the background.
disown	Forget a job.

All Linux shells have job control: the ability to run commands in the background (multitasking behind the scenes) and foreground (the active process at your shell prompt). A job is simply the shell’s unit of work. When you run a command interactively, your current shell tracks it as a job. When the command completes, the associated job disappears. Jobs are at a higher level than Linux processes; the Linux OS knows nothing about them. They are merely constructs of the shell. Here is some important vocabulary about job control:

Foreground job

In a shell, a running job that occupies the shell prompt so you cannot run another command

Background job

In a shell, a running job that doesn’t occupy the prompt, so you can run other commands in the same shell

Suspend

To stop a foreground job temporarily

Resume

To cause a suspended job to start running in the foreground again

Disown

To tell the shell to stop tracking the job; the underlying processes continue to run

---

jobs

stdin

stdout

- file

-- opt

--help

--version

The built-in command jobs lists the jobs running in your current shell by number and name:

→ jobs
[1]-  Running         emacs myfile &    A background job
[2]+  Stopped         ssh example.com   A suspended job

The integer on the left is the job number, and the plus sign identifies the default job affected by the fg (foreground) and bg (background) commands.

---

&

Placed at the end of a command line, the ampersand causes the given command to run as a background job:

→ emacs myfile &
[2] 28090

The shell’s response includes the job number (2) and the process ID of the command (28090).

---

^Z

Typing ^Z in a shell, while a job runs in the foreground, suspends that job. It simply stops running, but its state is remembered:

→ sleep 10                  Waits for 10 seconds
^Z
[1]+  Stopped    sleep 10
→

Now you’re ready to run bg to put the sleep command into the background, or fg to resume it in the foreground. You could also leave it suspended and run other commands.

---

suspend

stdin

stdout

- file

-- opt

--help

--version

The built-in suspend command pauses the current shell if possible, as if you’d applied ^Z to the shell itself. For instance, if you create a superuser shell with sudo and want to return to your original shell, suspend pauses the superuser shell:

→ whoami
smith
→ sudo bash                  Run a superuser shell
[sudo] password: xxxxxxxx
# whoami
root
# suspend                    Suspend the superuser shell
[1]+  Stopped    sudo bash
→ whoami                     Back to the original shell
smith

---

bg	stdin	stdout	- file	-- opt	--help	--version
bg [%job]

The built-in command bg sends a suspended job to run in the background. With no arguments, bg operates on the most recently suspended job. To specify a particular job (shown by the jobs command), supply the job number or name preceded by a percent sign:

→ bg %2          Send job 2 to the background
→ bg %cat        Send job beginning with “cat” to the background

Some types of interactive jobs cannot remain in the background—for instance, if they are waiting for input. If you try, the shell suspends the job and displays:

[2]+  Stopped      command line here

Now resume the job (with fg) and continue.

---

fg	stdin	stdout	- file	-- opt	--help	--version
fg [%job]

The built-in command fg brings a suspended or backgrounded job into the foreground. With no arguments, it selects a job, usually the most recently suspended or backgrounded one. To specify a particular job (as shown by the jobs command), supply the job number or name preceded by a percent sign:

→ fg %2      Bring job 2 into the foreground
→ fg %cat    Bring job beginning with “cat” into the foreground

---

disown	stdin	stdout	- file	-- opt	--help	--version
disown [-ar] [-h] [%job]

The built-in command disown tells your current shell to “forget” a job. The Linux processes behind the job keep running—you just can’t control them anymore with bg, fg, jobs, and other job-related commands. This is useful for long jobs that you don’t need to interact with, or jobs that should keep running after your shell exits. See also nohup in “Controlling Processes”.

→ disown %2         Forget job #2
→ disown %cat       Forget job beginning with “cat”
→ disown -h %2      Mark job #2 to keep running after shell exits
→ disown -r         Forget all running jobs
→ disown -a         Forget all jobs

Running Multiple Shells at Once

Job control can manage several commands at once, but only one can run in the foreground at a time. More powerfully, you can run multiple shells at once, each with a foreground command and any number of background commands.

If your Linux computer runs a window system such as KDE or GNOME, you can easily run many shells at the same time by opening multiple shell windows (see “Launching a Shell”). In addition, certain shell window programs, such as KDE’s konsole, can open multiple tabs within a single window, each one running a shell.

Even without a window system—say, over an SSH network connection—you can manage multiple shells at once. The tmux command simulates multiple shell windows in an ordinary ASCII terminal. Using special keystrokes, you can switch from one virtual window to another at will. (Another such program is screen, but tmux is better maintained and easier to configure.) To begin a session with tmux, run:

→ tmux

A new shell launches with an extra status bar at the bottom of the terminal, indicating that you’re running one virtual window. The tmux program provides 10 such windows by default, labeled from 0 to 9, that you may switch between. Each window runs a single shell at first, but you can split a window into multiple “panes” to display multiple shells at once. Try these keystrokes to get the hang of tmux:

1. In the current tmux window, run ls.

2. Press ^Bc (Ctrl-B, then press c). tmux displays a fresh shell prompt in a second virtual window. The status bar changes to show two virtual windows numbered 0 and 1.

3. In this second window, run a different command (say, df).

4. Press ^Bn and you’ll switch back to window 0, where your output from ls is now visible again.

5. Press ^Bn a few more times to toggle between the two virtual windows.

6. Press ^B% to split the current window into two panes side by side.

7. Press ^B" to split the current pane into two, vertically. You’re now viewing three shells in separate panes.

Most aspects of tmux are configurable in the file ~/.tmux_conf, even the choice of ^B as the prefix key. Here are common keystroke commands:

Keystroke	Meaning
^B?	Display online help. Press “q” to quit.
^Bc	Create a window.
^B0, ^B1 …​ ^B9	Switch to window 0 through 9, respectively.
^Bn	Switch to the next window, numerically.
^Bp	Switch to the previous window, numerically.
^Bl	Switch to the most recently used window.
^B%	Split into two panes side by side.
^B"	Split into two panes top and bottom.
^Bo	Jump to the next pane.
^B left arrow	Jump to the pane to the left.
^B right arrow	Jump to the pane to the right.
^B up arrow	Jump to the pane above.
^B down arrow	Jump to the pane below.
^Bq	Display pane numbers for reference.
^Bx	Kill the current pane.
^B^B	Send a true Ctrl-B to your shell, ignored by tmux.
^B^Z	Suspend tmux.
^Bd	“Detach” from a tmux session and return to your original shell. To return to tmux, run tmux attach.
^D	Terminate a shell in a window or pane. This is the ordinary “end of file” keystroke, explained in “Terminating a Shell”, which closes any shell.
^B:kill-session	Kill all windows and terminate tmux.

A few notes about running tmux:

• If shells within tmux are missing your aliases, variables, or other shell settings, that’s because tmux runs a login shell that does not source your .bashrc initialization file. It only sources your startup file (.bash_profile, .bash_login, or .profile). To correct this issue, append these lines to your startup file:

  # Source my .bashrc file
  if [ -f ~/.bashrc ]; then
    . ~/.bashrc
  fi

• If you run a text editor, tmux captures all Ctrl-B keystrokes, even those intended as editing commands. Press ^B^B to send a true Ctrl-B to your editor.

• Don’t run tmux locally on a graphical desktop; run multiple shell windows instead. It’s easier and avoids a problem: if you’ve configured your shell to run commands on logout (for example, in the file ~/.bash_logout), tmux’s shells will run those commands on exit, even though you haven’t logged out of the desktop. This may have unwanted effects on your desktop login session.

Killing a Command in Progress

To kill a foreground command immediately, press ^C. Here I kill the cat command as it prints a huge file:

→ cat hugefile
Lorem ipsum dolor sit amet, consectetur adipiscing
odio. Praesent libero. Sed cursus ante dapibus diam.
quis sem at nibh elementum blah blah blah ^C
→

To kill a background command, bring it into the foreground with fg and then press ^C:

→ sleep 50 &
[1] 12752
→ jobs
[1]-  Running       sleep 50 &
→ fg %1
sleep 50
^C
→

or run the kill command described in “Controlling Processes”. The keystroke ^C is a shell feature. It has no effect on programs that “catch” ^C and do not terminate, like text editors and GUI applications. Use kill for those.

Surviving a Kill

Killing a command with ^C may leave your shell in an odd or unresponsive state, because the killed program could not close itself properly. A common symptom is not displaying the keystrokes you type. To fix the shell, follow these steps:

1. Press ^J to get a shell prompt. This keystroke may work even if pressing Enter does not.

2. Type the command reset (even if the letters don’t appear while you type) and press ^J to run it. Your shell should return to normal.

Terminating a Shell

To terminate a shell, either run the exit command:

→ exit

or press ^D on a line by itself. The keystroke ^D sends an “end of file” signal to any program reading from standard input. This includes the shell itself.

Tailoring Shell Behavior

Several files in your home directory control the behavior of bash shells. The startup files .bash_profile, .bash_login, and .profile contain commands that run each time you log in. (Choose just one startup file and stick with it. I recommend .bash_profile because some other shells also use .profile.) Commands in the initialization file .bashrc run every time you launch an interactive shell, and commands in .bash_logout run each time you log out. All these files can set variables, run programs, print silly messages, or whatever you like. Other Linux shells use other configuration files as shown in Table 1-1.

Table 1-1. Shell configuration files in $HOME and when they are read

Shell	On login	By other interactive shells	On logout
bash	.bash_profile, .bash_login, .profile	.bashrc	.bash_logout
dash	.profile
fish	.config/fish/config.fish	.config/fish/config.fish
ksha	.profile, .kshrc	.kshrc
tcsh	.tcshrc, .cshrc, .login	.tcshrc, .cshrc
zsha	.zshenv, .zprofile, .zlogin	.zshenv, .zshrc	.zlogout
a To override these file paths with environment variables, see the manpage.

Other shell configuration files live in /etc for system-wide control; see the respective manual page, or manpage, for each shell. All these configuration files are examples of shell scripts: executable files that contain shell commands. I cover this feature in more detail in “Programming with Shell Scripts”.

Getting Help

If you need more information than this book provides, here are several ways to get more help:

Run the man command

The man command displays documentation for a given program. For example, to learn about counting words in a file with wc, run:

→ man wc

To search for manual pages (manpages) by keyword for a particular topic, use the -k option followed by the keyword:

→ man -k database

If the list of manpages is longer than the screen, pipe it to less to display it in pages (press q to quit):

→ man -k database | less

Run the info command

The info command is an extended, hypertext help system covering many Linux commands.

→ info ls

While info runs, some useful keystrokes are:

• To get help, press h

• To quit, press q

• To page forward and backward, use the space bar and Backspace key, respectively

• To jump between hyperlinks, press Tab

• To follow a hyperlink, press Enter

  If info has no documentation on a given command, it displays the command’s manpage. For a listing of available documentation, type info by itself. To learn how to navigate the info system, run info info.

Use the --help option (if any)

Many Linux commands respond to the option --help or -h by printing a short help message. Try:

→ wc --help

If the output is longer than the screen, pipe it into less:

→ wc --help | less

Examine the directory /usr/share/doc

This directory contains supporting documents for programs, usually organized by name and version. For example, files for the editor Emacs version 28 are likely found (depending on distro) in /usr/share/doc/emacs28.

Distro-specific websites

Most Linux distros have an official site with documentation, discussion forums, and other resources. Search the web for your distro name (e.g., “Ubuntu”) to find its website. The Arch Linux wiki is particularly informative regardless of your distro.

Linux help sites

Ask Linux questions at cunix.stakexchange.org, linuxquestions.org, itsfoss.community, and nixcraft.com.

Web search

To decipher a Linux error message, paste it into a search engine, verbatim, optionally surrounded by double quotes.

This concludes my basic overview of Linux and the shell. Now let’s turn to the specifics of Linux commands. The rest of the book lists and describes the most useful commands to work with files, processes, users, networking, multimedia, and more.

¹ GNOME, KDE, and other environments are built on a common windowing system that is either X or Wayland. To see which system you’re using, run the command echo $XDG_SESSION_TYPE.

² If you see an error message “command not found,” don’t worry: a command probably isn’t installed on your system. See “Installing Software Packages”.

³ User “silver,” who is listed twice, is running two interactive shells at once.

⁴ In Linux, all files and directories descend from the root. This is unlike Windows, in which different devices are accessed by drive letters.

⁵ Linux filenames are case-sensitive, so upper and lowercase letters are not equivalent.

⁶ Some distros no longer make these distinctions. Fedora, for example, makes /bin a symbolic link to /usr/bin.

⁷ Also see my follow-up book, Efficient Linux at the Command Line, to grow your skills.

⁸ Some setups use ~/.bash_aliases for this purpose.

⁹ For example, you can capture standard output in a file and still have standard error messages appear on screen.