Linux System Programming

Linux System Programming

by Robert Love
Released September 2007
Publisher(s): O'Reilly Media, Inc.
ISBN: 9780596009588

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Buy on Amazon
Start your free trial

Book description

This book is about writing software that makes the most effective use of the system you're running on -- code that interfaces directly with the kernel and core system libraries, including the shell, text editor, compiler, debugger, core utilities, and system daemons. The majority of both Unix and Linux code is still written at the system level, and Linux System Programming focuses on everything above the kernel, where applications such as Apache, bash, cp, vim, Emacs, gcc, gdb, glibc, ls, mv, and X exist.

Written primarily for engineers looking to program (better) at the low level, this book is an ideal teaching tool for any programmer. Even with the trend toward high-level development, either through web software (such as PHP) or managed code (C#), someone still has to write the PHP interpreter and the C# virtual machine. Linux System Programming gives you an understanding of core internals that makes for better code, no matter where it appears in the stack. Debugging high-level code often requires you to understand the system calls and kernel behavior of your operating system, too.

Key topics include:

  • An overview of Linux, the kernel, the C library, and the C compiler
  • Reading from and writing to files, along with other basic file I/O operations, including how the Linux kernel implements and manages file I/O
  • Buffer size management, including the Standard I/O library
  • Advanced I/O interfaces, memory mappings, and optimization techniques
  • The family of system calls for basic process management
  • Advanced process management, including real-time processes
  • File and directories-creating, moving, copying, deleting, and managing them
  • Memory management -- interfaces for allocating memory, managing the memory youhave, and optimizing your memory access
  • Signals and their role on a Unix system, plus basic and advanced signal interfaces
  • Time, sleeping, and clock management, starting with the basics and continuing through POSIX clocks and high resolution timers
With Linux System Programming, you will be able to take an in-depth look at Linux from both a theoretical and an applied perspective as you cover a wide range of programming topics.

Publisher resources

View/Submit Errata

Table of contents

  1. Foreword
  2. Preface
    1. Audience and Assumptions
    2. Contents of This Book
    3. Versions Covered in This Book
    4. Conventions Used in This Book
    5. Safari® Books Online
    6. Using Code Examples
    7. How to Contact Us
    8. Acknowledgments
  3. 1. Introduction and Essential Concepts
    1. System Programming
      1. System Calls
        1. Invoking system calls
      2. The C Library
      3. The C Compiler
    2. APIs and ABIs
      1. APIs
      2. ABIs
    3. Standards
      1. POSIX and SUS History
      2. C Language Standards
      3. Linux and the Standards
      4. This Book and the Standards
    4. Concepts of Linux Programming
      1. Files and the Filesystem
        1. Regular files
        2. Directories and links
        3. Hard links
        4. Symbolic links
        5. Special files
        6. Filesystems and namespaces
      2. Processes
        1. Threads
        2. Process hierarchy
      3. Users and Groups
      4. Permissions
      5. Signals
      6. Interprocess Communication
      7. Headers
      8. Error Handling
    5. Getting Started with System Programming
  4. 2. File I/O
    1. Opening Files
      1. The open( ) System Call
        1. Flags for open( )
      2. Owners of New Files
      3. Permissions of New Files
      4. The creat( ) Function
      5. Return Values and Error Codes
    2. Reading via read( )
      1. Return Values
      2. Reading All the Bytes
      3. Nonblocking Reads
      4. Other Error Values
      5. Size Limits on read( )
    3. Writing with write( )
      1. Partial Writes
      2. Append Mode
      3. Nonblocking Writes
      4. Other Error Codes
      5. Size Limits on write( )
      6. Behavior of write( )
    4. Synchronized I/O
      1. fsync( ) and fdatasync( )
        1. Return values and error codes
      2. sync( )
      3. The O_SYNC Flag
      4. O_DSYNC and O_RSYNC
    5. Direct I/O
    6. Closing Files
      1. Error Values
    7. Seeking with lseek( )
      1. Seeking Past the End of a File
      2. Error Values
      3. Limitations
    8. Positional Reads and Writes
      1. Error Values
    9. Truncating Files
    10. Multiplexed I/O
      1. select( )
        1. Return values and error codes
        2. select( ) example
        3. Portable sleeping with select( )
        4. pselect( )
      2. poll( )
        1. Return values and error codes
        2. poll( ) example
        3. ppoll( )
      3. poll( ) Versus select( )
    11. Kernel Internals
      1. The Virtual Filesystem
      2. The Page Cache
      3. Page Writeback
    12. Conclusion
  5. 3. Buffered I/O
    1. User-Buffered I/O
      1. Block Size
    2. Standard I/O
      1. File Pointers
    3. Opening Files
      1. Modes
    4. Opening a Stream via File Descriptor
    5. Closing Streams
      1. Closing All Streams
    6. Reading from a Stream
      1. Reading a Character at a Time
        1. Putting the character back
      2. Reading an Entire Line
        1. Reading arbitrary strings
      3. Reading Binary Data
    7. Writing to a Stream
      1. Writing a Single Character
      2. Writing a String of Characters
      3. Writing Binary Data
    8. Sample Program Using Buffered I/O
    9. Seeking a Stream
      1. Obtaining the Current Stream Position
    10. Flushing a Stream
    11. Errors and End-of-File
    12. Obtaining the Associated File Descriptor
    13. Controlling the Buffering
    14. Thread Safety
      1. Manual File Locking
      2. Unlocked Stream Operations
    15. Critiques of Standard I/O
    16. Conclusion
  6. 4. Advanced File I/O
    1. Scatter/Gather I/O
      1. readv( ) and writev( )
        1. Return values
        2. writev( ) example
        3. readv( ) example
        4. Implementation
    2. The Event Poll Interface
      1. Creating a New Epoll Instance
      2. Controlling Epoll
      3. Waiting for Events with Epoll
      4. Edge- Versus Level-Triggered Events
    3. Mapping Files into Memory
      1. mmap( )
        1. The page size
          1. sysconf( )
          2. getpagesize( )
          3. PAGE_SIZE
        2. Return values and error codes
        3. Associated signals
      2. munmap( )
      3. Mapping Example
      4. Advantages of mmap( )
      5. Disadvantages of mmap( )
      6. Resizing a Mapping
        1. Return values and error codes
      7. Changing the Protection of a Mapping
        1. Return values and error codes
      8. Synchronizing a File with a Mapping
        1. Return values and error codes
      9. Giving Advice on a Mapping
        1. Return values and error codes
    4. Advice for Normal File I/O
      1. The posix_fadvise( ) System Call
        1. Return values and error codes
      2. The readahead( ) System Call
        1. Return values and error codes
      3. Advice Is Cheap
    5. Synchronized, Synchronous, and Asynchronous Operations
      1. Asynchronous I/O
        1. Thread-based asynchronous I/O
    6. I/O Schedulers and I/O Performance
      1. Disk Addressing
      2. The Life of an I/O Scheduler
      3. Helping Out Reads
        1. The Deadline I/O Scheduler
        2. The Anticipatory I/O Scheduler
        3. The CFQ I/O Scheduler
        4. The Noop I/O Scheduler
      4. Selecting and Configuring Your I/O Scheduler
      5. Optimizing I/O Performance
        1. Scheduling I/O in user space
          1. Sorting by path
          2. Sorting by inode
          3. Sorting by physical block
    7. Conclusion
  7. 5. Process Management
    1. The Process ID
      1. Process ID Allocation
      2. The Process Hierarchy
      3. pid_t
      4. Obtaining the Process ID and Parent Process ID
    2. Running a New Process
      1. The Exec Family of Calls
        1. The rest of the family
        2. Error values
      2. The fork( ) System Call
        1. Copy-on-write
        2. vfork( )
    3. Terminating a Process
      1. Other Ways to Terminate
      2. atexit( )
      3. on_exit( )
      4. SIGCHLD
    4. Waiting for Terminated Child Processes
      1. Waiting for a Specific Process
      2. Even More Waiting Versatility
      3. BSD Wants to Play: wait3( ) and wait4( )
      4. Launching and Waiting for a New Process
      5. Zombies
    5. Users and Groups
      1. Real, Effective, and Saved User and Group IDs
      2. Changing the Real or Saved User or Group ID
      3. Changing the Effective User or Group ID
      4. Changing the User and Group IDs, BSD Style
      5. Changing the User and Group IDs, HP-UX Style
      6. Preferred User/Group ID Manipulations
      7. Support for Saved User IDs
      8. Obtaining the User and Group IDs
    6. Sessions and Process Groups
      1. Session System Calls
      2. Process Group System Calls
      3. Obsolete Process Group Functions
    7. Daemons
    8. Conclusion
  8. 6. Advanced Process Management
    1. Process Scheduling
      1. Big-Oh Notation
      2. Timeslices
      3. I/O- Versus Processor-Bound Processes
      4. Preemptive Scheduling
      5. Threading
    2. Yielding the Processor
      1. Legitimate Uses
      2. Yielding, Past and Present
    3. Process Priorities
      1. nice( )
      2. getpriority( ) and setpriority( )
      3. I/O Priorities
    4. Processor Affinity
      1. sched_getaffinity() and sched_setaffinity( )
    5. Real-Time Systems
      1. Hard Versus Soft Real-Time Systems
      2. Latency, Jitter, and Deadlines
      3. Linux’s Real-Time Support
      4. Linux Scheduling Policies and Priorities
        1. The first in, first out policy
        2. The round-robin policy
        3. The normal policy
        4. The batch scheduling policy
        5. Setting the Linux scheduling policy
          1. Error codes
      5. Setting Scheduling Parameters
        1. Error codes
        2. Determining the range of valid priorities
      6. sched_rr_get_interval( )
        1. Error codes
      7. Precautions with Real-Time Processes
      8. Determinism
        1. Prefaulting data and locking memory
        2. CPU affinity and real-time processes
    6. Resource Limits
      1. The Limits
        1. Default limits
      2. Setting and Retrieving Limits
        1. Error codes
  9. 7. File and Directory Management
    1. Files and Their Metadata
      1. The Stat Family
      2. Permissions
      3. Ownership
      4. Extended Attributes
        1. Keys and values
        2. Extended attribute namespaces
        3. Extended attribute operations
          1. Retrieving an extended attribute
          2. Setting an extended attribute
          3. Listing the extended attributes on a file
          4. Removing an extended attribute
    2. Directories
      1. The Current Working Directory
        1. Obtaining the current working directory
        2. Changing the current working directory
      2. Creating Directories
      3. Removing Directories
      4. Reading a Directory’s Contents
        1. Reading from a directory stream
        2. Closing the directory stream
        3. System calls for reading directory contents
    3. Links
      1. Hard Links
      2. Symbolic Links
      3. Unlinking
    4. Copying and Moving Files
      1. Copying
      2. Moving
    5. Device Nodes
      1. Special Device Nodes
      2. The Random Number Generator
    6. Out-of-Band Communication
    7. Monitoring File Events
      1. Initializing inotify
      2. Watches
        1. Adding a new watch
        2. Watch masks
      3. inotify Events
        1. Reading inotify events
          1. Advanced inotify events
        2. Linking together move events
      4. Advanced Watch Options
      5. Removing an inotify Watch
      6. Obtaining the Size of the Event Queue
      7. Destroying an inotify Instance
  10. 8. Memory Management
    1. The Process Address Space
      1. Pages and Paging
        1. Sharing and copy-on-write
      2. Memory Regions
    2. Allocating Dynamic Memory
      1. Allocating Arrays
      2. Resizing Allocations
      3. Freeing Dynamic Memory
      4. Alignment
        1. Allocating aligned memory
          1. Older interfaces
        2. Other alignment concerns
          1. Nonstandard types
          2. Playing with pointers
    3. Managing the Data Segment
    4. Anonymous Memory Mappings
      1. Creating Anonymous Memory Mappings
      2. Mapping /dev/zero
    5. Advanced Memory Allocation
      1. Fine-Tuning with malloc_usable_size( ) and malloc_trim( )
    6. Debugging Memory Allocations
      1. Obtaining Statistics
    7. Stack-Based Allocations
      1. Duplicating Strings on the Stack
      2. Variable-Length Arrays
    8. Choosing a Memory Allocation Mechanism
    9. Manipulating Memory
      1. Setting Bytes
      2. Comparing Bytes
      3. Moving Bytes
      4. Searching Bytes
      5. Frobnicating Bytes
    10. Locking Memory
      1. Locking Part of an Address Space
      2. Locking All of an Address Space
      3. Unlocking Memory
      4. Locking Limits
      5. Is a Page in Physical Memory?
    11. Opportunistic Allocation
      1. Overcommitting and OOM
  11. 9. Signals
    1. Signal Concepts
      1. Signal Identifiers
      2. Signals Supported by Linux
    2. Basic Signal Management
      1. Waiting for a Signal, Any Signal
      2. Examples
      3. Execution and Inheritance
      4. Mapping Signal Numbers to Strings
    3. Sending a Signal
      1. Permissions
      2. Examples
      3. Sending a Signal to Yourself
      4. Sending a Signal to an Entire Process Group
    4. Reentrancy
      1. Guaranteed-Reentrant Functions
    5. Signal Sets
      1. More Signal Set Functions
    6. Blocking Signals
      1. Retrieving Pending Signals
      2. Waiting for a Set of Signals
    7. Advanced Signal Management
      1. The siginfo_t Structure
      2. The Wonderful World of si_code
    8. Sending a Signal with a Payload
      1. Example
    9. Conclusion
  12. 10. Time
    1. Time’s Data Structures
      1. The Original Representation
      2. And Now, Microsecond Precision
      3. Even Better: Nanosecond Precision
      4. Breaking Down Time
      5. A Type for Process Time
    2. POSIX Clocks
      1. Time Source Resolution
    3. Getting the Current Time of Day
      1. A Better Interface
      2. An Advanced Interface
      3. Getting the Process Time
    4. Setting the Current Time of Day
      1. Setting Time with Precision
      2. An Advanced Interface for Setting the Time
    5. Playing with Time
    6. Tuning the System Clock
    7. Sleeping and Waiting
      1. Sleeping with Microsecond Precision
      2. Sleeping with Nanosecond Resolution
      3. An Advanced Approach to Sleep
      4. A Portable Way to Sleep
      5. Overruns
      6. Alternatives to Sleeping
    8. Timers
      1. Simple Alarms
      2. Interval Timers
      3. Advanced Timers
        1. Creating a timer
        2. Arming a timer
        3. Obtaining the expiration of a timer
        4. Obtaining the overrun of a timer
        5. Deleting a timer
  13. A. GCC Extensions to the C Language
    1. GNU C
    2. Inline Functions
    3. Suppressing Inlining
    4. Pure Functions
    5. Constant Functions
    6. Functions That Do Not Return
    7. Functions That Allocate Memory
    8. Forcing Callers to Check the Return Value
    9. Marking Functions As Deprecated
    10. Marking Functions As Used
    11. Marking Functions or Parameters As Unused
    12. Packing a Structure
    13. Increasing the Alignment of a Variable
    14. Placing Global Variables in a Register
    15. Branch Annotation
    16. Getting the Type of an Expression
    17. Getting the Alignment of a Type
    18. The Offset of a Member Within a Structure
    19. Obtaining the Return Address of a Function
    20. Case Ranges
    21. Void and Function Pointer Arithmetic
    22. More Portable and More Beautiful in One Fell Swoop
  14. Bibliography
  15. About the Author
  16. Colophon
  17. Copyright

Product information

  • Title: Linux System Programming
  • Author(s): Robert Love
  • Release date: September 2007
  • Publisher(s): O'Reilly Media, Inc.
  • ISBN: 9780596009588