Unix in a Nutshell | O'Reilly Media

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The Unix operating system originated at AT&T Bell Labs in the early 1970s. System V Release 4 (SVR4) came from USL (Unix System Laboratories) in the late 1980s. Unix source ownership is currently a matter of litigation in U.S. courts. Because Unix was able to run on different hardware from different vendors, developers were encouraged to modify Unix and distribute it as their own value-added version. Separate Unix traditions evolved as a result: USL's System V, the Berkeley Software Distribution (BSD, from the University of California, Berkeley), Xenix, etc.

SVR4, which was developed jointly by USL (then a division of AT&T) and Sun Microsystems, merged features from BSD and SVR3. This added about two dozen BSD commands (plus some new SVR4 commands) to the basic Unix command set. In addition, SVR4 provided a BSD Compatibility Package, a kind of "second string" command group. This package included some of the most fundamental BSD commands, and its purpose was to help users of BSD-derived systems make the transition to SVR4.

Today, the specification of what makes a system "Unix" is embodied primarily in the POSIX standard, an international standard based on System V and BSD. Commercial Unix systems, such as Solaris from Sun Microsystems, AIX from IBM, and HP-UX from Hewlett Packard, are standard-adhering direct descendants of the original Unix systems.

A number of other systems are "spiritual" descendents of Unix, even though they contain none of the original Unix source code. The most notable of these systems is GNU/Linux, which has seen a meteoric rise in popularity. However, a large number of systems derived from the 4.4-BSD-Lite distribution are also popular. All of these systems offer standards compliance and compatibility with SVR4 and earlier versions of BSD.

This edition of Unix in a Nutshell attempts to define the cross-section of features and commands that "make a Unix system Unix." To that end, it covers three of the most popular and representative systems now available.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Today, the specification of what makes a system "Unix" is embodied primarily in the POSIX standard, an international standard based on System V and BSD. Commercial Unix systems, such as Solaris from Sun Microsystems, AIX from IBM, and HP-UX from Hewlett Packard, are standard-adhering direct descendants of the original Unix systems.

A number of other systems are "spiritual" descendents of Unix, even though they contain none of the original Unix source code. The most notable of these systems is GNU/Linux, which has seen a meteoric rise in popularity. However, a large number of systems derived from the 4.4-BSD-Lite distribution are also popular. All of these systems offer standards compliance and compatibility with SVR4 and earlier versions of BSD.

This edition of Unix in a Nutshell attempts to define the cross-section of features and commands that "make a Unix system Unix." To that end, it covers three of the most popular and representative systems now available.

Solaris 10: Solaris 10 is a distributed computing environment from Sun Microsystems. Solaris includes the SunOS 5.10 operating system, plus additional features such as the Common Desktop Environment, GNOME, and Java tools. In addition, the kernel has received significant enhancement to support multiprocessor CPUs, multithreaded processes, kernel-level threads, and dynamic loading of device drivers and other kernel modules. Most of the user-level (and system administration) content comes from SVR4. As a result, Solaris 10 is based on SVR4 but contains additional BSD/SunOS features. To help in the transition from the old (largely BSD-based) SunOS, Solaris provides the BSD/SunOS Compatibility Package and the Binary Compatibility Package.

Sun has made binary versions of Solaris for the SPARC and Intel architectures available for "free," for noncommercial use. You pay only for the media, shipping, and handling, or you may download installation CD images. To find out more, see

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

If you wish to build programs from source code, you need a compiler. Almost all Unix applications are written in C or C++, with the majority still written in C. This section describes obtaining compilers for the three systems covered in this book.

Solaris 10 includes a Java compiler. Earlier versions of Solaris did not come with C or C++ compilers. You had to either buy compilers from Sun, from other third party vendors, or find a binary of some version of GCC for use in bootstrapping the latest version of GCC.

The final version of Solaris 10 now includes GCC (both C and C++ compilers) in /usr/sfw/bin. This is true for both the SPARC and Intel x86 versions. You thus have a choice: you may use the supplied GCC, or buy high-quality C and C++ compilers from Sun.

Besides GCC, a very large number of precompiled packages is available from http://www.sunfreeware.com/. You should see both the "Download/Install" and "FAQ" sections of that web site.

All the software from http://www.sunfreeware.com is in pkgadd format and is installable using that command. (See Chapter 6.) We recommend reading the details on the web site, which will always be up to date.

Note that many commands discussed in this book won't be on your system if all you've done is an end user install. If you can afford the disk space, do at least a developer install. This also installs many of the header files and libraries that you need in order to compile programs from source code.

For support issues and publicly released patches to Solaris, the web starting point is http://sunsolve.sun.com.

GNU/Linux systems usually install software development tools by default. If your system does not have compiler tools or

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Many of the programs listed in Chapter 2 are available in source code form from the Internet. For GNU/Linux and Mac OS X, you may be able to use a package manager to download and install the software (see Chapter 6). Similarly, for Solaris, you may be able to get a precompiled version of the program from http://www.sunfreeware.com/.

However, it's possible, particularly on a commercial Unix system, that you will want (or need) to download the source and build the program yourself if you don't have it, or if you wish to obtain the very latest version. This section outlines the conventional build process.

Most Internet software is written in C or C++. To compile it you will need a compiler. See the previous section for a discussion of where to get a compiler if you don't have one.

Today's programs usually use the GNU Project's Autoconf software suite for managing portability issues. Autoconf generates a shell script named configure, which tests various aspects of the target system. The end result of running configure is a Makefile custom-tuned to the particular system (see Chapter 16), and a header file describing the features available, or missing, from the system. As a result, the recipe for building software is usually quite simple, consisting of the following:

Download the software. This can be done with a noninteractive program such as wget or curl (see their entries in Chapter 2), or interactively using anonymous FTP for programs distributed that way.
Decompress and extract the software.
Change directory into the program's distribution directory.
Run configure.
Run make.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This guide presents the major features of Solaris, GNU/Linux, and Mac OS X. In addition, this guide presents chapters on Emacs, RCS, CVS, Subversion, GNU Make, and GDB, the GNU debugger. Although they are not part of commercial Unix systems, they are found on many Unix systems because they are useful add-ons.

But keep in mind: if your system doesn't include all the component packages, there will be commands in this book you won't find on your system.

The summary of Unix commands in Chapter 2 makes up a large part of this book. Only user/programmer commands are included; administrative commands are purposely ignored. Chapter 2 describes the following set:

Commands and options in Solaris, GNU/Linux, and Mac OS X. This includes many "essential" tools for which source and/or binaries are available via the Internet
Solaris-only tools
GNU/Linux-only tools
Mac OS X-only tools
Java-related tools

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

If you're just beginning to work on a Unix system, the abundance of commands might prove daunting. To help orient you, the following lists present a small sampling of commands on various topics.

ftp	Interactive file transfer program.
login	Sign on to Unix.
mailx	Read or send mail.
slogin	Sign on to remote Unix using secure shell.
ssh	Connect to another system, securely.

cmp	Compare two files, byte by byte.
comm	Compare items in two sorted files.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Where the behavior specified by the POSIX standard differs from the historical behavior provided by a command, Solaris provides a different version of the command in either /usr/xpg6/bin or in /usr/xpg4/bin. On Solaris systems, you should place these two directories into your search path before the standard /usr/bin directory. Some of these commands are not covered in this book, since they are either administrative commands or are obsolete. Also, today, it is unlikely that the commands in /usr/ucb will be useful; you probably should not have that directory in your search path.

ar	delta	file	kill	nm	tail
awk	df	find	link	nohup

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This chapter presents the Unix commands of interest to users and programmers. Most of these commands appear in the "Commands" section of the online manual. With rare exception, this book purposely avoids system administration commands, because system administration is beyond its scope. The focus instead is on everyday commands, those used both interactively and for programming.

Summarizing three operating systems that are similar but not identical is a daunting task. In order to make a coherent presentation, the chapter is organized as follows:

Common Commands: This section lists commands that should be available on just about any Unix system. We have included here many commands that are downloadable from the Internet and that are standard with GNU/Linux, such as autoconf or wget, even though they may not come "out of the box" on commercial Unix systems. Wherever possible, we provide a URL from which the source to the command may be downloaded, so that you can build the program yourself if you want it. See the section "Obtaining Compilers" in Chapter 1 for what to do if you don't have a C compiler for your system.

Additionally, we have made an effort to be as concise as possible. For example, GNU-style long options are listed side-by-side with their standard single-letter counterparts. Similarly, several commands have associated with them additional more specialized commands that are needed only rarely. We simply list such commands as "related," without giving them separate entries. For such commands, you should then see your system's online manual pages or other documentation.
Solaris Commands: This section lists the important commands that are available only on Solaris.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

This chapter presents the Unix commands of interest to users and programmers. Most of these commands appear in the "Commands" section of the online manual. With rare exception, this book purposely avoids system administration commands, because system administration is beyond its scope. The focus instead is on everyday commands, those used both interactively and for programming.

Summarizing three operating systems that are similar but not identical is a daunting task. In order to make a coherent presentation, the chapter is organized as follows:

Common Commands: This section lists commands that should be available on just about any Unix system. We have included here many commands that are downloadable from the Internet and that are standard with GNU/Linux, such as autoconf or wget, even though they may not come "out of the box" on commercial Unix systems. Wherever possible, we provide a URL from which the source to the command may be downloaded, so that you can build the program yourself if you want it. See the section "Obtaining Compilers" in Chapter 1 for what to do if you don't have a C compiler for your system.

Additionally, we have made an effort to be as concise as possible. For example, GNU-style long options are listed side-by-side with their standard single-letter counterparts. Similarly, several commands have associated with them additional more specialized commands that are needed only rarely. We simply list such commands as "related," without giving them separate entries. For such commands, you should then see your system's online manual pages or other documentation.
Solaris Commands: This section lists the important commands that are available only on Solaris.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

cde

Common Desktop Environment

The Common Desktop Environment (CDE) is one of the graphical user interfaces (GUI) on Solaris systems. Solaris users may choose between CDE and GNOME.

Documenting CDE would require its own book and is beyond the scope of this one. Instead, listed here are some of the more useful individual CDE commands, which are kept in /usr/dt/bin. (Commands for the Desktop.) In addition, a number of OpenWindows commands are still useful. See the listing under openwin .

Useful CDE Programs

The following CDE and Sun Desktop commands may be of interest. Check the manpages for more information.

answerbook2

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

GNU/Linux programs generally accept --help and --version options. In the interest of brevity, the individual command descriptions omit listing those options.

aspell

    aspell [options] [files]

aspell is intended to be a drop-in replacement for ispell, but with more functionality. It thus accepts the same options; see ispell for more information. See also spell in the earlier section "Alphabetical Summary of Common Commands."

URL: http://aspell.net/ and http://www.gnu.org/software/aspell/.

cdda2wav

    cdda2wav [options] [output.wav]

Convert Compact Disc Digital Audio (CDDA) to the WAV format. This process is often called "ripping" a CD-ROM, and is generally performed before using an encoder to convert the file to a compressed music format such as OGG or MP3. By default,

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

apply

    apply [options] command argument ...

Apply command to one or more of the following arguments in turn. command may contain a % followed by a digit from 1 to 9. Such text is replaced with the corresponding following unused argument.

Options

- N: Use arguments in groups of N. For example, -2 uses two arguments for each invocation of command. Occurrences of % N in command override this option.
-a char: Use char instead of % as the special character to represent arguments.
-d: Display the commands that would be executed, but don't actually execute them.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

appletviewer

    appletviewer [options] urls

Connect to the specified urls and run any Java applets they specify in their own windows, outside the context of a web browser.

Options

-debug: Run the applet viewer from within the Java debugger, jdb.
-encoding name: Specify the input HTML file encoding.
-J java-option: Pass java-option on to the java program. Useful for changing the execution environment or memory usage. java-option should not contain spaces; use multiple -J options if necessary.

apt

    apt [options] [files ...] [@file ...]

Annotation processing tool. The

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

For novice users, this chapter presents basic concepts about the Unix shell. For advanced users, this chapter also summarizes the major similarities and differences between the Bash, Korn, and "Tenex" C shells . Details on the three shells are provided in Chapters 4 and 5.

The following topics are presented:

Introduction to the shell
Purpose of the shell
Shell flavors
Shell source code
Common features
Differing features

Today's microwave ovens, and many other household appliances (ovens, washing machines, dishwashers), let you use simple push-buttons to instruct them what to do. They provide a simple user interface to a possibly complicated internal system.

The shell is the user interface to Unix, and by the same token, several shells are available in Unix. Most systems provide more than one for you to choose from. Each shell has different features, but all of them affect how commands will be interpreted and provide tools to create your Unix environment.

The original shells were developed before the time of Graphical User Interfaces (GUIs), and at first glance, appear harder to use than GUI interfaces. The truth, though, is that they aren't harder to use, they are harder to learn. However, once you've mastered them, you'll find that you can accomplish an infinite variety of tasks that just cannot be managed with a GUI.

The shell is simply a program that allows the system to understand your commands. (That's why the shell is often called a command interpreter.) For many users, the shell works invisibly--"behind the scenes." Your only concern is that the system does what you tell it to do; you don't care about the inner workings. In our microwave analogy, this is comparable to pressing the START button. Most of us don't care whether the user interface communicates with an embedded microcomputer, or drives analog electronics, as long as the popcorn is ready in time for the movie, and doesn't burn.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Today's microwave ovens, and many other household appliances (ovens, washing machines, dishwashers), let you use simple push-buttons to instruct them what to do. They provide a simple user interface to a possibly complicated internal system.

The shell is the user interface to Unix, and by the same token, several shells are available in Unix. Most systems provide more than one for you to choose from. Each shell has different features, but all of them affect how commands will be interpreted and provide tools to create your Unix environment.

The original shells were developed before the time of Graphical User Interfaces (GUIs), and at first glance, appear harder to use than GUI interfaces. The truth, though, is that they aren't harder to use, they are harder to learn. However, once you've mastered them, you'll find that you can accomplish an infinite variety of tasks that just cannot be managed with a GUI.

The shell is simply a program that allows the system to understand your commands. (That's why the shell is often called a command interpreter.) For many users, the shell works invisibly--"behind the scenes." Your only concern is that the system does what you tell it to do; you don't care about the inner workings. In our microwave analogy, this is comparable to pressing the START button. Most of us don't care whether the user interface communicates with an embedded microcomputer, or drives analog electronics, as long as the popcorn is ready in time for the movie, and doesn't burn.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

There are three uses for the shell:

Interactive use
Customization of your Unix session
Programming

When the shell is used interactively, the system waits for you to type a command at the Unix prompt. Your commands can include special symbols that let you abbreviate filenames or redirect input and output.

A Unix shell defines variables to control the behavior of your Unix session. Setting these variables tells the system, for example, which directory to use as your home directory, or the file in which you store your mail. Some variables are preset by the system; you can define others in startup files that are read when you log in. Startup files can also contain Unix commands or special shell commands. These are executed every time you log in. Many shells also support special variables and internal commands that let you tailor the behavior of the shell itself.

Unix shells provide a set of special (or built-in) commands that let you create programs called shell scripts . In fact, many built-in commands can be used interactively like Unix commands, and Unix commands are frequently used in shell scripts. Scripts are useful for executing a series of individual commands. This is similar to BATCH files in MS-DOS and Windows. Scripts can also execute commands repeatedly (in a loop) or conditionally (if-else), as in many high-level programming languages.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Many different Unix shells are available. This quick reference describes the three most popular shells:

The GNU Project's Bash (Bourne-Again SHell), arguably the most popular shell in use today. It is a superset of the original Bourne shell , including command-line editing and many features first implemented in the Korn shell.
The Korn shell, a superset of the original Bourne shell that lets you edit the command line. There are two commonly available versions of the Korn shell, distinguished by the year they were released, and referred to in this book as ksh88 and ksh93 respectively.
The "Tenex" C shell, an enhanced version of the original BSD C shell, which uses C-like syntax and is more convenient for the interactive user than the original Bourne shell.

The original Bourne shell is available as /bin/sh on commercial Unix systems, and if invoked as sh, Bash will do its best to emulate the original Bourne shell's behavior. However, it is rare today to find the original Bourne shell being used interactively as a login shell; other shells that provide better interactive features and the Bourne shell's programming language, such as Bash and ksh, are more popular. However, when writing shell scripts, most people are careful to restrict themselves to just those features of the Bourne shell.

The /etc/passwd file determines which shell takes effect during your interactive Unix session. When you log in, the system checks your entry in /etc/passwd. The last field of each entry names a program to run as the default shell. For example:

If the program name is:	Your shell is the:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Here is a list of URLs for the source code for different shells.

Shell	Location
Bash	`ftp://ftp.gnu.org/gnu/bash`
Bash source code patches	`ftp://ftp.gnu.org/gnu/bash/bash-3.0-patches`
Ksh93	`http://www.research.att.com/sw/download/`
The Z Shell	`http://www.zsh.org`
The Public Domain Korn Shell	`http://web.cs.mun.ca/~michael/pdksh/`
Tcsh	`http://www.tcsh.org`

The Public Domain Korn shell is mostly compatible with ksh88, and is usually the version of ksh shipped with GNU/Linux systems. The Z shell (zsh) has many features of the Bourne shell, Bash, and the Korn shell, and a plethora of features unique to it.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The following table displays features that are common to the Bash, Korn, and C shells. Note that both the Korn shell and Bash are enhanced versions of the Bourne shell; therefore, they include all features of the Bourne shell, plus some others.

Symbol/command	Meaning/action
^a Brace expansion is a compile-time feature in the Korn shell. Usually commercial versions don't have it, but if you compile from source code, you do get it by default.
`>`	Redirect output.
`>>`	Append to file.
`<`	Redirect input.
`<<`	"Here" document (redirect input).
`\|`	Pipe output.
`&`	Run process in background.
`;`	Separate commands on same line.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The following table displays features that are different among the three shells.

bash	ksh	tcsh	Meaning/action
`$`	`$`	`%`	Prompt.
`>\|`	`>\|`	`>!`	Force redirection.
		`>>!`	Force append.
`>` file `2>&1`	`>` file `2>&1`	`>&` file

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The original Bourne shell distributed with V7 Unix in 1979 became the standard shell for writing shell scripts. The Bourne shell is still to be found in /bin/sh on many commercial Unix systems. The Bourne shell itself has not changed that much since its initial release, although it has seen modest enhancements over the years. The most notable new features were the CDPATH variable and a built-in test command with System III (circa 1980), command hashing and shell functions for System V Release 2 (circa 1984), and the addition of job control features for System V Release 4 (1989).

Because the Berkeley C shell (csh) offered features that were more pleasant for interactive use, such as command history and job control, for a long time the standard practice in the Unix world was to use the Bourne shell for programming and the C shell for daily use. David Korn at Bell Labs was the first developer to enhance the Bourne shell by adding csh-like features to it: history, job control, and additional programmability. Eventually, the Korn shell's feature set surpassed both the Bourne shell and the C shell, while remaining compatible with the Bourne shell for shell programming. Today, the POSIX standard defines the "standard shell" language and behavior based on the System V Bourne shell, with a selected subset of features from the Korn shell.

On most commercial Unix systems, including Solaris, /bin/ksh is ksh88 . On Mac OS X (10.4 and newer), however, it's a recent version of ksh93 from AT&T Research.

The Free Software Foundation, in keeping with its goal to produce a complete Unix work-alike system, developed a clone of the Bourne shell, written from scratch, named "Bash," the Bourne-Again SHell. Over time, Bash has become a POSIX-compliant version of the shell, with many additional features. A large part of these additional features overlap the features of the Korn shell, but Bash is not an exact Korn shell clone.

This chapter covers Bash , which is the primary shell for GNU/Linux and Mac OS X. Because

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·

Table of Contents