Beginning Shell Scripting

Beginning Shell Scripting

by John C. Welch, Eric Foster-Johnson, Micah Anderson
Released April 2005
Publisher(s): Wrox
ISBN: 9780764583209

Read it now on the O’Reilly learning platform with a 10-day free trial.

O’Reilly members get unlimited access to books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Book description

  • Covering all major platforms-Linux, Unix, Mac OS X, and Windows-this guide shows programmers and power users how to customize an operating system, automate commands, and simplify administration tasks using shell scripts

  • Offers complete shell-scripting instructions, robust code examples, and full scripts for OS customization

  • Covers shells as a user interface, basic scripting techniques, script editing and debugging, graphing data, and simplifying administrative tasks

  • In addition to Unix and Linux scripting, the book covers the latest Windows scripting techniques and offers a complete tutorial on Mac OS X scripting, including detailed coverage of mobile file systems, legacy applications, Mac text editors, video captures, and the Mac OS X Open Scripting Architecture

    •

    Table of contents

    1. Copyright
    2. About the Authors
    3. Credits
    4. Introduction
      1. Whom This Book Is For
      2. What Does This Book Cover?
      3. How This Book Is Structured
      4. What Do You Need to Use This Book?
      5. Conventions
        1. Source Code
      6. Errata
      7. p2p.wrox.com
    5. 1. Introducing Shells
      1. 1.1. What Is a Shell?
      2. 1.2. Why Use Shells?
      3. 1.3. What Kind of Shells Are There?
        1. 1.3.1. The Bourne Shell
        2. 1.3.2. The C Shell
        3. 1.3.3. The Korn Shell
        4. 1.3.4. Bash, the Bourne Again Shell
        5. 1.3.5. tcsh, the T C Shell
        6. 1.3.6. Other Shells
        7. 1.3.7. Graphical Shells
        8. 1.3.8. The Official POSIX Shell
        9. 1.3.9. Default Shells
        10. 1.3.10. Choosing a Shell
        11. 1.3.11. Changing Your Default Shell
        12. 1.3.12. Running a Shell from Within Another Shell
        13. 1.3.13. Finding Out More About the Shell
      4. 1.4. How Do Shells Fit into a Graphical Environment?
        1. 1.4.1. Running Shells on Linux
          1. 1.4.1.1. Running the GNOME Shell Window
          2. 1.4.1.2. Running the KDE Shell Window
          3. 1.4.1.3. Active Select and Paste
        2. 1.4.2. Running Shells on Mac OS X
        3. 1.4.3. Running Shells on Unix Systems
        4. 1.4.4. Running Shells on Windows—Isn't command.com Enough?
          1. 1.4.4.1. Installing Cygwin on Windows
          2. 1.4.4.2. Installing the Korn Shell on Windows
        5. 1.4.5. Running Shells on PDAs and Other Systems
      5. 1.5. Entering Commands
        1. 1.5.1. Determining Which Shell You Are Running
        2. 1.5.2. Command-Line Options
          1. 1.5.2.1. Combining Options
          2. 1.5.2.2. Strange Options
      6. 1.6. Command Editing
        1. 1.6.1. Command Substitution
          1. 1.6.1.1. Repeating Previous Commands
          2. 1.6.1.2. Repeating Parts of Previous Commands
          3. 1.6.1.3. Using the Up and Down Arrows
        2. 1.6.2. Viewing the Command History
        3. 1.6.3. Calling Up an Editor
        4. 1.6.4. Using File-Name Completion
      7. 1.7. Working with Wildcards
        1. 1.7.1. The * Wildcard
        2. 1.7.2. The ? Wildcard
      8. 1.8. Running Commands in the Background
      9. 1.9. Summary
    6. 2. Introducing Shell Scripts
      1. 2.1. What Are Shell Scripts?
      2. 2.2. Alternatives to Shells: Other Scripting Languages
        1. 2.2.1. Perl
        2. 2.2.2. Python
        3. 2.2.3. Tcl
        4. 2.2.4. MS-DOS Batch Files
      3. 2.3. Tools to Edit Shell Scripts
        1. 2.3.1. Using Legacy Editors
          1. 2.3.1.1. Delving into emacs
          2. 2.3.1.2. Learning vi Modes
        2. 2.3.2. Moving to Graphical Text Editors
          1. 2.3.2.1. Cross-Platform Editors
          2. 2.3.2.2. Mac OS X Editors
          3. 2.3.2.3. Linux Editors
          4. 2.3.2.4. Unix Editors
          5. 2.3.2.5. Microsoft Windows Editors
      4. 2.4. Writing Scripts
        1. 2.4.1. Remembering Commands So You Don't Have To
        2. 2.4.2. Outputting Text
        3. 2.4.3. Variables
          1. 2.4.3.1. Variables and the C Shell
          2. 2.4.3.2. Combining Variables
          3. 2.4.3.3. Building Up Commands with Variables
          4. 2.4.3.4. De-assigning Variables
        4. 2.4.4. Gathering Input
        5. 2.4.5. Commenting Your Scripts
        6. 2.4.6. Continuing Lines
      5. 2.5. Summary
      6. 2.6. Exercises
    7. 3. Controlling How Scripts Run
      1. 3.1. Referencing Variables
      2. 3.2. Looping and Iteration
        1. 3.2.1. Looping over Files
        2. 3.2.2. Looping for a Fixed Number of Iterations
        3. 3.2.3. Looping Like a C Program—the bash Shell
        4. 3.2.4. Looping in the C Shell
        5. 3.2.5. Nested Loops
      3. 3.3. Checking Conditions with if
        1. 3.3.1. Or Else, What?
        2. 3.3.2. What Is Truth?
        3. 3.3.3. Redirecting Output
        4. 3.3.4. Using elif (Short for else if)
      4. 3.4. Nesting if Statements
      5. 3.5. Testing with the test Command
        1. 3.5.1. Comparing Numbers
        2. 3.5.2. Comparing Text Strings
        3. 3.5.3. Testing Files
        4. 3.5.4. Using the Binary and Not Operators
        5. 3.5.5. Creating Shorthand Tests with [
      6. 3.6. Making Complex Decisions with case
        1. 3.6.1. Handling Problematic Input
        2. 3.6.2. Using case with the C Shell
      7. 3.7. Looping While a Condition Is True
      8. 3.8. Looping Until a Condition Is True
      9. 3.9. Summary
      10. 3.10. Exercises
    8. 4. Interacting with the Environment
      1. 4.1. Examining Environment Variables
        1. 4.1.1. Reading Environment Variables
          1. 4.1.1.1. Handling Local and Environment Variables
          2. 4.1.1.2. Listing the Environment with the C Shell
          3. 4.1.1.3. Testing the Environment
        2. 4.1.2. Setting Environment Variables
          1. 4.1.2.1. Setting Environment Variables in the C Shell
          2. 4.1.2.2. Reading Values into the Current Environment
      2. 4.2. Customizing Your Account
        1. 4.2.1. How the Bourne Shell Starts Up
        2. 4.2.2. How the Korn Shell Starts Up
        3. 4.2.3. How the C Shell Starts Up
        4. 4.2.4. How the T C Shell Starts Up
        5. 4.2.5. How Bash Starts Up
      3. 4.3. Handling Command-Line Arguments
        1. 4.3.1. Reading Command-Line Arguments with the Bourne Shell
        2. 4.3.2. Reading Command-Line Arguments with the C Shell
      4. 4.4. Making Scripts Executable
        1. 4.4.1. Marking Files Executable
        2. 4.4.2. Setting the #! Magic Line
      5. 4.5. Summary
      6. 4.6. Exercises
    9. 5. Scripting with Files
      1. 5.1. Combining Files into Archives
      2. 5.2. Working with File Modes
      3. 5.3. Testing Files with the test Command
      4. 5.4. Dealing with Mac OS X Files
        1. 5.4.1. The Legacies of NeXT
        2. 5.4.2. Mobile File Systems and Mac OS X
          1. 5.4.2.1. Target Disk Mode
          2. 5.4.2.2. Mobile File Systems and Disk Images
        3. 5.4.3. Naming Issues
      5. 5.5. HFS+ Versus UFS: The Mac OS X Holy War
        1. 5.5.1. The Terror of the Resource Fork
      6. 5.6. Working with Here Files and Interactive Programs
        1. 5.6.1. Displaying Messages with Here Files
        2. 5.6.2. Customizing Here Files
        3. 5.6.3. Driving Interactive Programs with Here Files
        4. 5.6.4. Turning Off Variable Substitution
      7. 5.7. Summary
      8. 5.8. Exercises
    10. 6. Processing Text with sed
      1. 6.1. Introducing sed
      2. 6.2. sed Versions
      3. 6.3. Installing sed
        1. 6.3.1. Bootstrap Installation
        2. 6.3.2. Configuring and Installing sed
      4. 6.4. How sed Works
        1. 6.4.1. Invoking sed
        2. 6.4.2. Editing Commands
        3. 6.4.3. Invoking sed with the -e Flag
        4. 6.4.4. The -n, --quiet, and --silent Flags
        5. 6.4.5. sed Errors
      5. 6.5. Selecting Lines to Operate On
        1. 6.5.1. Address Ranges
        2. 6.5.2. Address Negation
        3. 6.5.3. Address Steps
      6. 6.6. Substitution
        1. 6.6.1. Substitution Flags
        2. 6.6.2. Using an Alternative String Separator
        3. 6.6.3. Address Substitution
      7. 6.7. Advanced sed Invocation
        1. 6.7.1. The comment Command
        2. 6.7.2. The insert, append, and change Commands
      8. 6.8. Advanced Addressing
        1. 6.8.1. Regular Expression Addresses
        2. 6.8.2. Character Class Keywords
        3. 6.8.3. Regular Expression Address Ranges
        4. 6.8.4. Combining Line Addresses with regexps
      9. 6.9. Advanced Substitution
        1. 6.9.1. Referencing Matched regexps with &
        2. 6.9.2. Back References
      10. 6.10. Hold Space
      11. 6.11. More sed Resources
        1. 6.11.1. Common One-Line sed Scripts
        2. 6.11.2. Common sed Commands
        3. 6.11.3. Less Common sed Commands
        4. 6.11.4. GNU sed-Specific sed Extensions
      12. 6.12. Summary
      13. 6.13. Exercises
    11. 7. Processing Text with awk
      1. 7.1. What Is awk (Gawk/Mawk/Nawk/Oawk)?
        1. 7.1.1. Gawk, the GNU awk
        2. 7.1.2. What Version Do I Have Installed?
        3. 7.1.3. Installing gawk
      2. 7.2. How awk Works
      3. 7.3. Invoking awk
      4. 7.4. The print Command
        1. 7.4.1. Using Field Separators
        2. 7.4.2. Using the printf Command
        3. 7.4.3. Using printf Format Modifiers
        4. 7.4.4. Using the sprintf Command
      5. 7.5. Using Variables in awk
        1. 7.5.1. User-Defined Variables
        2. 7.5.2. Built-in Variables
          1. 7.5.2.1. The FS Variable
          2. 7.5.2.2. FS Regular Expressions
          3. 7.5.2.3. The NR Variable
      6. 7.6. Control Statements
        1. 7.6.1. if Statements
        2. 7.6.2. Comparison Operators
        3. 7.6.3. Arithmetic Functions
        4. 7.6.4. Output Redirection
        5. 7.6.5. While Loops
        6. 7.6.6. For Loops
      7. 7.7. Functions
      8. 7.8. Resources
      9. 7.9. Summary
      10. 7.10. Exercises
    12. 8. Creating Command Pipelines
      1. 8.1. Working with Standard Input and Output
        1. 8.1.1. Redirecting Standard Input and Output
        2. 8.1.2. Redirecting Standard Error
        3. 8.1.3. Redirecting Both Standard Output and Standard Error
        4. 8.1.4. Appending to Files
        5. 8.1.5. Truncating Files
        6. 8.1.6. Sending Output to Nowhere Fast
      2. 8.2. Piping Commands
        1. 8.2.1. Piping with Unix Commands
        2. 8.2.2. Creating Pipelines
        3. 8.2.3. Using tee to Send the Output to More Than One Process
      3. 8.3. Summary
      4. 8.4. Exercises
    13. 9. Controlling Processes
      1. 9.1. Exploring Processes
        1. 9.1.1. Checking Process IDs
        2. 9.1.2. Reading the /proc File System
        3. 9.1.3. Killing Processes
      2. 9.2. Launching Processes
        1. 9.2.1. Running Commands in the Foreground
        2. 9.2.2. Running Commands in the Background
        3. 9.2.3. Running Commands in Subshells
        4. 9.2.4. Running Commands with the exec Command
      3. 9.3. Capturing the Output of Processes
        1. 9.3.1. Using Backticks for Command Substitution
          1. 9.3.1.1. Using Parentheses in Place of Backticks
          2. 9.3.1.2. Setting Variables from Commands
          3. 9.3.1.3. Performing Math with Commands
          4. 9.3.1.4. Reading Files into Variables
        2. 9.3.2. Capturing Program Return Codes
      4. 9.4. Summary
      5. 9.5. Exercises
    14. 10. Shell Scripting Functions
      1. 10.1. Defining Functions
        1. 10.1.1. Adding Names to Blocks of Code
        2. 10.1.2. Function Declaration Errors
      2. 10.2. Using Functions
        1. 10.2.1. Declaring before Use
        2. 10.2.2. Function Files
        3. 10.2.3. Common Usage Errors
        4. 10.2.4. Undeclaring Functions
      3. 10.3. Using Arguments with Functions
      4. 10.4. Using Return Codes with Functions
      5. 10.5. Variable Scope: Think Globally, Act Locally
      6. 10.6. Understanding Recursion
      7. 10.7. Summary
      8. 10.8. Exercises
    15. 11. Debugging Shell Scripts
      1. 11.1. Deciphering Error Messages
        1. 11.1.1. Finding Missing Syntax
        2. 11.1.2. Finding Syntax Errors
      2. 11.2. Tracking Down Problems with Debugging Techniques
        1. 11.2.1. Look Backward
        2. 11.2.2. Look for Obvious Mistakes
        3. 11.2.3. Look for Weird Things
        4. 11.2.4. Look for Hidden Assumptions
        5. 11.2.5. Divide and Conquer
        6. 11.2.6. Break the Script into Pieces
        7. 11.2.7. Trace the Execution
        8. 11.2.8. Get Another Set of Eyes
      3. 11.3. Running Scripts in Debugging Mode
        1. 11.3.1. Disabling the Shell
        2. 11.3.2. Displaying the Script Commands
        3. 11.3.3. Combining the -n and -v Options
        4. 11.3.4. Tracing Script Execution
      4. 11.4. Avoiding Errors with Good Scripting
        1. 11.4.1. Tidy Up Your Scripts
        2. 11.4.2. Comment Your Scripts
        3. 11.4.3. Create Informative Error Messages
        4. 11.4.4. Simplify Yourself Out of the Box
        5. 11.4.5. Test, Test, and Test Again
      5. 11.5. Summary
      6. 11.6. Exercises
    16. 12. Graphing Data with MRTG
      1. 12.1. Working with MRTG
      2. 12.2. Monitoring Other Data with MRTG
      3. 12.3. Installing MRTG
      4. 12.4. Writing Scripts for MRTG
      5. 12.5. Configuring MRTG
        1. 12.5.1. Configuring the Global Values
        2. 12.5.2. Configuring MRTG Targets for Your Scripts
        3. 12.5.3. Customizing MRTG Output
          1. 12.5.3.1. Configuring Target HTML Outputs
          2. 12.5.3.2. Configuring Graphs
      6. 12.6. Running MRTG
        1. 12.6.1. Viewing Your First MRTG Output
        2. 12.6.2. Configuring cron
        3. 12.6.3. Maximizing MRTG Performance
      7. 12.7. Monitoring Your Computer with MRTG
        1. 12.7.1. Graphing Memory Usage
        2. 12.7.2. Graphing CPU Usage
        3. 12.7.3. Graphing Disk Usage
      8. 12.8. Monitoring Networks with MRTG
      9. 12.9. Monitoring Applications with MRTG
      10. 12.10. Summary
      11. 12.11. Exercises
    17. 13. Scripting for Administrators
      1. 13.1. Why Write Scripts?
      2. 13.2. Scripting Complicated Commands
      3. 13.3. Troubleshooting Your Systems
      4. 13.4. Removing Minor Annoyances
      5. 13.5. Cleaning Up Data
      6. 13.6. Automating Daily Work
      7. 13.7. Summary
      8. 13.8. Exercises
    18. 14. Scripting for the Desktop
      1. 14.1. Scripting Office Applications
        1. 14.1.1. Scripting the OpenOffice.org Suite
          1. 14.1.1.1. Scripting OpenOffice with the Shell
          2. 14.1.1.2. Scripting OpenOffice.org in Basic
        2. 14.1.2. Scripting AbiWord
        3. 14.1.3. Scripting NEdit
      2. 14.2. Scripting for the Desktop on Mac OS X
        1. 14.2.1. Open Scripting Architecture
        2. 14.2.2. AppleScript Basics
          1. 14.2.2.1. AppleScript Dictionaries
          2. 14.2.2.2. AppleScript's Shell Support
            1. 14.2.2.2.1. Going from Shell to AppleScript
            2. 14.2.2.2.2. Going from AppleScript to Shell
        3. 14.2.3. Mac OS X Terminal Window Settings
      3. 14.3. Scripting Multimedia
        1. 14.3.1. Scripting the XMMS Music Player
        2. 14.3.2. Scripting Rhythmbox
        3. 14.3.3. Scripting the Totem Movie Player
      4. 14.4. Scripting Other Desktop Applications
      5. 14.5. Where to Go from Here
      6. 14.6. Summary
      7. 14.7. Exercises
    19. A. Answers to Exercises
      1. A.1.
        1. A.1.1.
          1. A.1.1.1. Chapter 2
          2. A.1.1.2. Chapter 3
          3. A.1.1.3. Chapter 4
          4. A.1.1.4. Chapter 5
          5. A.1.1.5. Chapter 6
          6. A.1.1.6. Chapter 7
          7. A.1.1.7. Chapter 8
          8. A.1.1.8. Chapter 9
          9. A.1.1.9. Chapter 10
          10. A.1.1.10. Chapter 11
          11. A.1.1.11. Chapter 12
          12. A.1.1.12. Chapter 13
          13. A.1.1.13. Chapter 14
    20. B. Useful Commands
      1. B.1. Navigating the System
        1. B.1.1. exit
          1. B.1.1.1. Description
          2. B.1.1.2. Example
          3. B.1.1.3. Options
        2. B.1.2. file
          1. B.1.2.1. Description
          2. B.1.2.2. Examples
          3. B.1.2.3. Options
        3. B.1.3. kill
          1. B.1.3.1. Description
          2. B.1.3.2. Examples
          3. B.1.3.3. Options
        4. B.1.4. man
          1. B.1.4.1. Description
          2. B.1.4.2. Example
          3. B.1.4.3. Options
        5. B.1.5. nohup
          1. B.1.5.1. Description
          2. B.1.5.2. Example
          3. B.1.5.3. Options
        6. B.1.6. printenv
          1. B.1.6.1. Description
          2. B.1.6.2. Example
          3. B.1.6.3. Options
        7. B.1.7. ps
          1. B.1.7.1. Description
          2. B.1.7.2. Examples
          3. B.1.7.3. Options
        8. B.1.8. sleep
          1. B.1.8.1. Description
          2. B.1.8.2. Examples
          3. B.1.8.3. Options
        9. B.1.9. type
          1. B.1.9.1. Description
          2. B.1.9.2. Examples
          3. B.1.9.3. Options
        10. B.1.10. uname
          1. B.1.10.1. Description
          2. B.1.10.2. Examples
          3. B.1.10.3. Options
        11. B.1.11. who
          1. B.1.11.1. Description
          2. B.1.11.2. Examples
          3. B.1.11.3. Options
        12. B.1.12. whoami
          1. B.1.12.1. Description
          2. B.1.12.2. Examples
          3. B.1.12.3. Options
      2. B.2. Working with Files and Directories
        1. B.2.1. basename
          1. B.2.1.1. Description
          2. B.2.1.2. Examples
          3. B.2.1.3. Options
        2. B.2.2. cat
          1. B.2.2.1. Description
          2. B.2.2.2. Examples
          3. B.2.2.3. Options
        3. B.2.3. chmod
          1. B.2.3.1. Description
          2. B.2.3.2. Examples
          3. B.2.3.3. Options
        4. B.2.4. chown
          1. B.2.4.1. Description
          2. B.2.4.2. Example
          3. B.2.4.3. Options
        5. B.2.5. cp
          1. B.2.5.1. Description
          2. B.2.5.2. Examples
          3. B.2.5.3. Options
        6. B.2.6. df
          1. B.2.6.1. Description
          2. B.2.6.2. Examples
          3. B.2.6.3. Options
        7. B.2.7. du
          1. B.2.7.1. Description
          2. B.2.7.2. Examples
          3. B.2.7.3. Options
        8. B.2.8. find
          1. B.2.8.1. Description
          2. B.2.8.2. Examples
          3. B.2.8.3. Options
        9. B.2.9. grep
          1. B.2.9.1. Description
          2. B.2.9.2. Examples
          3. B.2.9.3. Options
        10. B.2.10. head
          1. B.2.10.1. Description
          2. B.2.10.2. Examples
          3. B.2.10.3. Options
        11. B.2.11. ls
          1. B.2.11.1. Description
          2. B.2.11.2. Examples
          3. B.2.11.3. Options
        12. B.2.12. mkdir
          1. B.2.12.1. Description
          2. B.2.12.2. Examples
          3. B.2.12.3. Options
        13. B.2.13. mv
          1. B.2.13.1. Description
          2. B.2.13.2. Examples
          3. B.2.13.3. Options
        14. B.2.14. rm
          1. B.2.14.1. Description
          2. B.2.14.2. Examples
          3. B.2.14.3. Options
        15. B.2.15. rmdir
          1. B.2.15.1. Description
          2. B.2.15.2. Examples
          3. B.2.15.3. Option
        16. B.2.16. tail
          1. B.2.16.1. Description
          2. B.2.16.2. Examples
          3. B.2.16.3. Options
        17. B.2.17. touch
          1. B.2.17.1. Description
          2. B.2.17.2. Examples
          3. B.2.17.3. Options
      3. B.3. Manipulating Text
        1. B.3.1. awk
          1. B.3.1.1. Description
          2. B.3.1.2. Example
          3. B.3.1.3. Options
        2. B.3.2. echo
          1. B.3.2.1. Description
          2. B.3.2.2. Examples
          3. B.3.2.3. Options
      4. B.4. Transforming Data
        1. B.4.1. cut
          1. B.4.1.1. Description
          2. B.4.1.2. Example
          3. B.4.1.3. Options
        2. B.4.2. sed
          1. B.4.2.1. Description
          2. B.4.2.2. Example
          3. B.4.2.3. Options
        3. B.4.3. sort
          1. B.4.3.1. Description
          2. B.4.3.2. Examples
          3. B.4.3.3. Options
        4. B.4.4. strings
          1. B.4.4.1. Description
          2. B.4.4.2. Example
          3. B.4.4.3. Options
        5. B.4.5. tr
          1. B.4.5.1. Description
          2. B.4.5.2. Example
          3. B.4.5.3. Options
      5. B.5. Resolving Expressions
        1. B.5.1. bc
          1. B.5.1.1. Description
          2. B.5.1.2. Example
          3. B.5.1.3. Options
        2. B.5.2. expr
          1. B.5.2.1. Description
          2. B.5.2.2. Examples
          3. B.5.2.3. Options

    Product information

    • Title: Beginning Shell Scripting
    • Author(s): John C. Welch, Eric Foster-Johnson, Micah Anderson
    • Release date: April 2005
    • Publisher(s): Wrox
    • ISBN: 9780764583209