How Linux Works, 2nd Edition

How Linux Works, 2nd Edition

by Brian Ward
Released November 2014
Publisher(s): No Starch Press
ISBN: 9781593275679

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on ebooks.com
Start your free trial

Book description

The "hows" and "whys" of How Linux Works: for administrators, programmers, and enthusiasts who want to understand the inner workings of the best operating system.

Table of contents

  1. How Linux Works: What Every Superuser Should Know
  2. Praise for the First Edition of How Linux Works
  3. Preface
    1. Who Should Read This Book?
    2. Prerequisites
    3. How to Read This Book
    4. A Hands-On Approach
    5. How This Book is Organized
    6. What’s New in the Second Edition?
    7. A Note on Terminology
  4. Acknowledgments
  5. 1. The Big Picture
    1. 1.1 Levels and Layers of Abstraction in a Linux System
    2. 1.2 Hardware: Understanding Main Memory
    3. 1.3 The Kernel
      1. 1.3.1 Process Management
      2. 1.3.2 Memory Management
      3. 1.3.3 Device Drivers and Management
      4. 1.3.4 System Calls and Support
    4. 1.4 User Space
    5. 1.5 Users
    6. 1.6 Looking Forward
  6. 2. Basic Commands and Directory Hierarchy
    1. 2.1 The Bourne Shell: /bin/sh
    2. 2.2 Using the Shell
      1. 2.2.1 The Shell Window
      2. 2.2.2 cat
      3. 2.2.3 Standard Input and Standard Output
    3. 2.3 Basic Commands
      1. 2.3.1 ls
      2. 2.3.2 cp
      3. 2.3.3 mv
      4. 2.3.4 touch
      5. 2.3.5 rm
      6. 2.3.6 echo
    4. 2.4 Navigating Directories
      1. 2.4.1 cd
      2. 2.4.2 mkdir
      3. 2.4.3 rmdir
      4. 2.4.4 Shell Globbing (Wildcards)
    5. 2.5 Intermediate Commands
      1. 2.5.1 grep
      2. 2.5.2 less
      3. 2.5.3 pwd
      4. 2.5.4 diff
      5. 2.5.5 file
      6. 2.5.6 find and locate
      7. 2.5.7 head and tail
      8. 2.5.8 sort
    6. 2.6 Changing Your Password and Shell
    7. 2.7 Dot Files
    8. 2.8 Environment and Shell Variables
    9. 2.9 The Command Path
    10. 2.10 Special Characters
    11. 2.11 Command-Line Editing
    12. 2.12 Text Editors
    13. 2.13 Getting Online Help
    14. 2.14 Shell Input and Output
      1. 2.14.1 Standard Error
      2. 2.14.2 Standard Input Redirection
    15. 2.15 Understanding Error Messages
      1. 2.15.1 Anatomy of a UNIX Error Message
      2. 2.15.2 Common Errors
        1. No such file or directory
        2. File exists
        3. Not a directory, Is a directory
        4. No space left on device
        5. Permission denied
        6. Operation not permitted
        7. Segmentation fault, Bus error
    16. 2.16 Listing and Manipulating Processes
      1. 2.16.1 Command Options
      2. 2.16.2 Killing Processes
      3. 2.16.3 Job Control
      4. 2.16.4 Background Processes
    17. 2.17 File Modes and Permissions
      1. 2.17.1 Modifying Permissions
      2. 2.17.2 Symbolic Links
      3. 2.17.3 Creating Symbolic Links
    18. 2.18 Archiving and Compressing Files
      1. 2.18.1 gzip
      2. 2.18.2 tar
        1. Unpacking tar files
        2. Table-of-Contents Mode
      3. 2.18.3 Compressed Archives (.tar.gz)
      4. 2.18.4 zcat
      5. 2.18.5 Other Compression Utilities
    19. 2.19 Linux Directory Hierarchy Essentials
      1. 2.19.1 Other Root Subdirectories
      2. 2.19.2 The /usr Directory
      3. 2.19.3 Kernel Location
    20. 2.20 Running Commands as the Superuser
      1. 2.20.1 sudo
      2. 2.20.2 /etc/sudoers
    21. 2.21 Looking Forward
  7. 3. Devices
    1. 3.1 Device Files
    2. 3.2 The sysfs Device Path
    3. 3.3 dd and Devices
    4. 3.4 Device Name Summary
      1. 3.4.1 Hard Disks: /dev/sd*
      2. 3.4.2 CD and DVD Drives: /dev/sr*
      3. 3.4.3 PATA Hard Disks: /dev/hd*
      4. 3.4.4 Terminals: /dev/tty*, /dev/pts/*, and /dev/tty
        1. Display Modes and Virtual Consoles
      5. 3.4.5 Serial Ports: /dev/ttyS*
      6. 3.4.6 Parallel Ports: /dev/lp0 and /dev/lp1
      7. 3.4.7 Audio Devices: /dev/snd/*, /dev/dsp, /dev/audio, and More
      8. 3.4.8 Creating Device Files
    5. 3.5 udev
      1. 3.5.1 devtmpfs
      2. 3.5.2 udevd Operation and Configuration
      3. 3.5.3 udevadm
      4. 3.5.4 Monitoring Devices
    6. 3.6 In-Depth: SCSI and the Linux Kernel
      1. 3.6.1 USB Storage and SCSI
      2. 3.6.2 SCSI and ATA
      3. 3.6.3 Generic SCSI Devices
      4. 3.6.4 Multiple Access Methods for a Single Device
  8. 4. Disks and Filesystems
    1. 4.1 Partitioning Disk Devices
      1. 4.1.1 Viewing a Partition Table
        1. Initial Kernel Read
      2. 4.1.2 Changing Partition Tables
      3. 4.1.3 Disk and Partition Geometry
      4. 4.1.4 Solid-State Disks (SSDs)
    2. 4.2 Filesystems
      1. 4.2.1 Filesystem Types
      2. 4.2.2 Creating a Filesystem
      3. 4.2.3 Mounting a Filesystem
      4. 4.2.4 Filesystem UUID
      5. 4.2.5 Disk Buffering, Caching, and Filesystems
      6. 4.2.6 Filesystem Mount Options
        1. Short Options
        2. Long Options
      7. 4.2.7 Remounting a Filesystem
      8. 4.2.8 The /etc/fstab Filesystem Table
      9. 4.2.9 Alternatives to /etc/fstab
      10. 4.2.10 Filesystem Capacity
      11. 4.2.11 Checking and Repairing Filesystems
        1. Checking ext3 and ext4 Filesystems
        2. The Worst Case
      12. 4.2.12 Special-Purpose Filesystems
    3. 4.3 swap space
      1. 4.3.1 Using a Disk Partition as Swap Space
      2. 4.3.2 Using a File as Swap Space
      3. 4.3.3 How Much Swap Do You Need?
    4. 4.4 Looking Forward: Disks and User Space
    5. 4.5 Inside a Traditional Filesystem
      1. 4.5.1 Viewing Inode Details
      2. 4.5.2 Working with Filesystems in User Space
      3. 4.5.3 The Evolution of Filesystems
  9. 5. How the Linux Kernel Boots
    1. 5.1 Startup Messages
    2. 5.2 Kernel Initialization and Boot Options
    3. 5.3 Kernel Parameters
    4. 5.4 Boot Loaders
      1. 5.4.1 Boot Loader Tasks
      2. 5.4.2 Boot Loader Overview
    5. 5.5 GRUB Introduction
      1. 5.5.1 Exploring Devices and Partitions with the GRUB Command Line
        1. Listing Devices
        2. File Navigation
      2. 5.5.2 GRUB Configuration
        1. Reviewing Grub.cfg
        2. Generating a New Configuration File
      3. 5.5.3 GRUB Installation
        1. Installing GRUB on Your System
        2. Installing GRUB on an External Storage Device
        3. Installing GRUB with UEFI
    6. 5.6 UEFI Secure Boot Problems
    7. 5.7 Chainloading Other Operating Systems
    8. 5.8 Boot Loader Details
      1. 5.8.1 MBR Boot
      2. 5.8.2 UEFI Boot
      3. 5.8.3 How GRUB Works
  10. 6. How User Space Starts
    1. 6.1 Introduction to init
    2. 6.2 System V Runlevels
    3. 6.3 Identifying Your init
    4. 6.4 systemd
      1. 6.4.1 Units and Unit Types
      2. 6.4.2 systemd Dependencies
        1. Ordering
        2. Conditional Dependencies
      3. 6.4.3 systemd Configuration
        1. Unit Files
        2. Enabling Units and the [Install] Section
        3. Variables and Specifiers
      4. 6.4.4 systemd Operation
      5. 6.4.5 Adding Units to systemd
        1. Removing Units
      6. 6.4.6 systemd Process Tracking and Synchronization
      7. 6.4.7 systemd On-Demand and Resource-Parallelized Startup
        1. Boot Optimization with Auxiliary Units
        2. An Example Socket Unit and Service
        3. Instances and Handoff
      8. 6.4.8 systemd System V Compatibility
      9. 6.4.9 systemd Auxiliary Programs
    5. 6.5 Upstart
      1. 6.5.1 Upstart Initialization Procedure
      2. 6.5.2 Upstart Jobs
        1. Viewing Jobs
        2. Job State Transitions
      3. 6.5.3 Upstart Configuration
        1. A Service Job: tty1
        2. Process Tracking and the Upstart expect Stanza
      4. 6.5.4 Upstart Operation
      5. 6.5.5 Upstart Logs
      6. 6.5.6 Upstart Runlevels and System V Compatibility
    6. 6.6 System V init
      1. respawn
      2. ctrlaltdel
      3. sysinit
      4. 6.6.1 System V init: Startup Command Sequence
      5. 6.6.2 The System V init Link Farm
        1. Starting and Stopping Services
        2. Modifying the Boot Sequence
      6. 6.6.3 run-parts
      7. 6.6.4 Controlling System V init
    7. 6.7 Shutting Down Your System
    8. 6.8 The Initial RAM Filesystem
    9. 6.9 Emergency Booting and Single-User Mode
  11. 7. System Configuration: Logging, System Time, Batch Jobs, and Users
    1. 7.1 The Structure of /etc
    2. 7.2 System Logging
      1. 7.2.1 The System Logger
      2. 7.2.2 Configuration Files
        1. Facility and Priority
        2. Extended Syntax
        3. Troubleshooting
        4. Logging: Past and Future
    3. 7.3 User Management Files
      1. 7.3.1 The /etc/passwd File
      2. 7.3.2 Special Users
      3. 7.3.3 The /etc/shadow File
      4. 7.3.4 Manipulating Users and Passwords
        1. Changing /etc/passwd as the Superuser
      5. 7.3.5 Working with Groups
    4. 7.4 getty and login
    5. 7.5 Setting the Time
      1. 7.5.1 Kernel Time Representation and Time Zones
      2. 7.5.2 Network Time
    6. 7.6 Scheduling Recurring Tasks with cron
      1. 7.6.1 Installing Crontab Files
      2. 7.6.2 System Crontab Files
      3. 7.6.3 The Future of cron
    7. 7.7 Scheduling One-Time Tasks with at
    8. 7.8 Understanding User IDs and User Switching
      1. 7.8.1 Process Ownership, Effective UID, Real UID, and Saved UID
        1. Typical Setuid Program Behavior
        2. Security Implications
    9. 7.9 User Identification and Authentication
      1. 7.9.1 Using Libraries for User Information
    10. 7.10 PAM
      1. 7.10.1 PAM Configuration
        1. Function Types
        2. Control Arguments and Stacked Rules
        3. Module Arguments
      2. 7.10.2 Notes on PAM
      3. 7.10.3 PAM and Passwords
    11. 7.11 Looking Forward
  12. 8. A Closer Look at Processes and Resource Utilization
    1. 8.1 Tracking Processes
    2. 8.2 Finding Open Files with lsof
      1. 8.2.1 Reading the lsof Output
      2. 8.2.2 Using lsof
    3. 8.3 Tracing Program Execution and System Calls
      1. 8.3.1 strace
      2. 8.3.2 ltrace
    4. 8.4 Threads
      1. 8.4.1 Single-Threaded and Multithreaded Processes
      2. 8.4.2 Viewing Threads
    5. 8.5 Introduction to Resource Monitoring
    6. 8.6 Measuring CPU Time
    7. 8.7 Adjusting Process Priorities
    8. 8.8 Load Averages
      1. 8.8.1 Using uptime
      2. 8.8.2 High Loads
    9. 8.9 Memory
      1. 8.9.1 How Memory Works
      2. 8.9.2 Page Faults
        1. Minor Page Faults
        2. Major Page Faults
        3. Watching Page Faults
    10. 8.10 Monitoring CPU and Memory Performance with vmstat
    11. 8.11 I/O Monitoring
      1. 8.11.1 Using iostat
      2. 8.11.2 Per-Process I/O Utilization and Monitoring: iotop
    12. 8.12 Per-Process Monitoring with pidstat
    13. 8.13 Further Topics
  13. 9. Understanding your Network and its Configuration
    1. 9.1 Network Basics
      1. 9.1.1 Packets
    2. 9.2 Network Layers
    3. 9.3 The Internet Layer
      1. 9.3.1 Viewing Your Computer’s IP Addresses
      2. 9.3.2 Subnets
      3. 9.3.3 Common Subnet Masks and CIDR Notation
    4. 9.4 Routes and the Kernel Routing Table
      1. 9.4.1 The Default Gateway
    5. 9.5 Basic ICMP and DNS Tools
      1. 9.5.1 ping
      2. 9.5.2 traceroute
      3. 9.5.3 DNS and host
    6. 9.6 The Physical Layer and Ethernet
    7. 9.7 Understanding Kernel Network Interfaces
    8. 9.8 Introduction to Network Interface Configuration
      1. 9.8.1 Manually Adding and Deleting Routes
    9. 9.9 Boot-Activated Network Configuration
    10. 9.10 Problems with Manual and Boot-Activated Network Configuration
    11. 9.11 Network Configuration Managers
      1. 9.11.1 NetworkManager Operation
      2. 9.11.2 Interacting with NetworkManager
      3. 9.11.3 NetworkManager Configuration
        1. Unmanaged Interfaces
        2. Dispatching
    12. 9.12 Resolving Hostnames
      1. 9.12.1 /etc/hosts
      2. 9.12.2 resolv.conf
      3. 9.12.3 Caching and Zero-Configuration DNS
      4. 9.12.4 /etc/nsswitch.conf
    13. 9.13 Localhost
    14. 9.14 The Transport Layer: TCP, UDP, and Services
      1. 9.14.1 TCP Ports and Connections
      2. 9.14.2 Establishing TCP Connections
      3. 9.14.3 Port Numbers and /etc/services
      4. 9.14.4 Characteristics of TCP
      5. 9.14.5 UDP
    15. 9.15 Revisiting a Simple Local Network
    16. 9.16 Understanding DHCP
      1. 9.16.1 The Linux DHCP Client
      2. 9.16.2 Linux DHCP Servers
    17. 9.17 Configuring Linux as a Router
      1. 9.17.1 Internet Uplinks
    18. 9.18 Private Networks
    19. 9.19 Network Address Translation (IP Masquerading)
    20. 9.20 Routers and Linux
    21. 9.21 Firewalls
      1. 9.21.1 Linux Firewall Basics
      2. 9.21.2 Setting Firewall Rules
      3. 9.21.3 Firewall Strategies
    22. 9.22 Ethernet, IP, and ARP
    23. 9.23 Wireless Ethernet
      1. 9.23.1 iw
      2. 9.23.2 Wireless Security
    24. 9.24 Summary
  14. 10. Network Applications and Services
    1. 10.1 The Basics of Services
      1. 10.1.1 A Closer Look
    2. 10.2 Network Servers
    3. 10.3 Secure Shell (SSH)
      1. 10.3.1 The SSHD Server
        1. Host Keys
        2. Starting the SSH Server
      2. 10.3.2 The SSH Client
        1. SSH File Transfer Clients
        2. SSH Clients for Non-Unix Platforms
    4. 10.4 The inetd and xinetd Daemons
      1. 10.4.1 TCP Wrappers: tcpd, /etc/hosts.allow, and /etc/hosts.deny
    5. 10.5 Diagnostic Tools
      1. 10.5.1 lsof
        1. Filtering by Protocol and Port
        2. Filtering by Connection Status
      2. 10.5.2 tcpdump
        1. Primitives
        2. Operators
        3. When Not to Use tcpdump
      3. 10.5.3 netcat
      4. 10.5.4 Port Scanning
    6. 10.6 Remote Procedure Call (RPC)
    7. 10.7 Network Security
      1. 10.7.1 Typical Vulnerabilities
      2. 10.7.2 Security Resources
    8. 10.8 Looking Forward
    9. 10.9 Sockets: How Processes Communicate with the Network
    10. 10.10 Unix Domain Sockets
      1. 10.10.1 Advantages for Developers
      2. 10.10.2 Listing Unix Domain Sockets
  15. 11. Introduction to Shell Scripts
    1. 11.1 Shell Script Basics
      1. 11.1.1 Limitations of Shell Scripts
    2. 11.2 Quoting and Literals
      1. 11.2.1 Literals
      2. 11.2.2 Single Quotes
      3. 11.2.3 Double Quotes
      4. 11.2.4 Passing a Literal Single Quote
    3. 11.3 Special Variables
      1. 11.3.1 Individual Arguments: $1, $2, ...
      2. 11.3.2 Number of Arguments: $#
      3. 11.3.3 All Arguments: $@
      4. 11.3.4 Script Name: $0
      5. 11.3.5 Process ID: $$
      6. 11.3.6 Exit Code: $?
    4. 11.4 Exit Codes
    5. 11.5 Conditionals
      1. 11.5.1 Getting Around Empty Parameter Lists
      2. 11.5.2 Using Other Commands for Tests
      3. 11.5.3 elif
      4. 11.5.4 && and || Logical Constructs
      5. 11.5.5 Testing Conditions
        1. File Tests
        2. String Tests
        3. Arithmetic Tests
      6. 11.5.6 Matching Strings with case
    6. 11.6 Loops
      1. 11.6.1 for Loops
      2. 11.6.2 while Loops
    7. 11.7 Command Substitution
    8. 11.8 Temporary File Management
    9. 11.9 Here Documents
    10. 11.10 Important Shell Script Utilities
      1. 11.10.1 basename
      2. 11.10.2 awk
      3. 11.10.3 sed
      4. 11.10.4 xargs
      5. 11.10.5 expr
      6. 11.10.6 exec
    11. 11.11 Subshells
    12. 11.12 Including Other Files in Scripts
    13. 11.13 Reading User Input
    14. 11.14 When (Not) to Use Shell Scripts
  16. 12. Moving Files Across the Network
    1. 12.1 Quick Copy
    2. 12.2 rsync
      1. 12.2.1 rsync Basics
      2. 12.2.2 Making Exact Copies of a Directory Structure
      3. 12.2.3 Using the Trailing Slash
      4. 12.2.4 Excluding Files and Directories
      5. 12.2.5 Transfer Integrity, Safeguards, and Verbose Modes
      6. 12.2.6 Compression
      7. 12.2.7 Limiting Bandwidth
      8. 12.2.8 Transferring Files to Your Computer
      9. 12.2.9 Further rsync Topics
    3. 12.3 Introduction to File Sharing
    4. 12.4 Sharing Files with Samba
      1. 12.4.1 Configuring the Server
      2. 12.4.2 Server Access Control
      3. 12.4.3 Passwords
        1. Adding and Deleting Users
        2. Changing Passwords
      4. 12.4.4 Starting the Server
      5. 12.4.5 Diagnostics and Log Files
      6. 12.4.6 Configuring a File Share
      7. 12.4.7 Home Directories
      8. 12.4.8 Sharing Printers
      9. 12.4.9 Using the Samba Client
      10. 12.4.10 Accessing Files as a Client
        1. Using the CIFS Filesystem
    5. 12.5 NFS Clients
    6. 12.6 Further Network File Service Options and Limitations
  17. 13. User Environments
    1. 13.1 Guidelines for Creating Startup Files
    2. 13.2 When to Alter Startup Files
    3. 13.3 Shell Startup File Elements
      1. 13.3.1 The Command Path
        1. Adding Dot (.) to the Path
      2. 13.3.2 The Manual Page Path
      3. 13.3.3 The Prompt
      4. 13.3.4 Aliases
      5. 13.3.5 The Permissions Mask
    4. 13.4 Startup File Order and Examples
      1. 13.4.1 The bash Shell
        1. Login Shells
        2. Non-Login Shells
        3. The Consequences of Two Kinds of Shells
        4. Example .bashrc
        5. Checking for Login and Interactive Shells
      2. 13.4.2 The tcsh Shell
        1. Example .cshrc
    5. 13.5 Default User Settings
      1. 13.5.1 Shell Defaults
      2. 13.5.2 Editor
      3. 13.5.3 Pager
    6. 13.6 Startup File Pitfalls
    7. 13.7 Further Startup Topics
  18. 14. A Brief Survey of the Linux Desktop
    1. 14.1 Desktop Components
      1. 14.1.1 Window Managers
      2. 14.1.2 Toolkits
      3. 14.1.3 Desktop Environments
      4. 14.1.4 Applications
    2. 14.2 A Closer Look at the X Window System
      1. 14.2.1 Display Managers
      2. 14.2.2 Network Transparency
    3. 14.3 Exploring X Clients
      1. 14.3.1 X Events
      2. 14.3.2 Understanding X Input and Preference Settings
        1. Input Devices (General)
        2. Mouse
        3. Keyboard
        4. Desktop Background
        5. xset
    4. 14.4 The Future of X
    5. 14.5 D-Bus
      1. 14.5.1 System and Session Instances
      2. 14.5.2 Monitoring D-Bus Messages
    6. 14.6 Printing
      1. 14.6.1 CUPS
      2. 14.6.2 Format Conversion and Print Filters
    7. 14.7 Other Desktop Topics
  19. 15. Development Tools
    1. 15.1 The C Compiler
      1. 15.1.1 Multiple Source Files
      2. 15.1.2 Header (Include) Files and Directories
        1. Fixing Include File Problems
        2. What Is the C Preprocessor (cpp)?
      3. 15.1.3 Linking with Libraries
      4. 15.1.4 Shared Libraries
        1. Listing Shared Library Dependencies
        2. How ld.so Finds Shared Libraries
        3. Linking Programs Against Shared Libraries
        4. Problems with Shared Libraries
    2. 15.2 make
      1. 15.2.1 A Sample Makefile
      2. 15.2.2 Built-in Rules
      3. 15.2.3 Final Program Build
      4. 15.2.4 Staying Up-to-Date
      5. 15.2.5 Command-Line Arguments and Options
      6. 15.2.6 Standard Macros and Variables
      7. 15.2.7 Conventional Targets
      8. 15.2.8 Organizing a Makefile
    3. 15.3 Debuggers
    4. 15.4 Lex and Yacc
    5. 15.5 Scripting Languages
      1. 15.5.1 Python
      2. 15.5.2 Perl
      3. 15.5.3 Other Scripting Languages
    6. 15.6 Java
    7. 15.7 Looking Forward: Compiling Packages
  20. 16. Introduction to Compiling Software From C Source Code
    1. 16.1 Software Build Systems
    2. 16.2 Unpacking C Source Packages
      1. 16.2.1 Where to Start
    3. 16.3 GNU Autoconf
      1. 16.3.1 An Autoconf Example
      2. 16.3.2 Installing Using a Packaging Tool
      3. 16.3.3 configure Script Options
        1. Using Separate Build Directories
      4. 16.3.4 Environment Variables
      5. 16.3.5 Autoconf Targets
      6. 16.3.6 Autoconf Log Files
      7. 16.3.7 pkg-config
        1. How pkg-config Works
        2. Installing pkg-config Files in Nonstandard Locations
    4. 16.4 Installation Practice
      1. 16.4.1 Where to Install
    5. 16.5 Applying a Patch
    6. 16.6 Troubleshooting Compiles and Installations
      1. 16.6.1 Specific Errors
    7. 16.7 Looking Forward
  21. 17. Building on the Basics
    1. 17.1 Web Servers and Applications
    2. 17.2 Databases
      1. 17.2.1 Types of Databases
    3. 17.3 Virtualization
    4. 17.4 Distributed and On-Demand Computing
    5. 17.5 Embedded Systems
    6. 17.6 Final Remarks
  22. A. Bibliography
  23. B. Updates
  24. Index
  25. About the Author
  26. Copyright

Product information

  • Title: How Linux Works, 2nd Edition
  • Author(s): Brian Ward
  • Release date: November 2014
  • Publisher(s): No Starch Press
  • ISBN: 9781593275679