Linux Command Directory


Linux in a Nutshell

This directory of Linux commands is from Linux in a Nutshell, 5th Edition.

Click on any of the 687 commands below to get a description and list of available options. All links in the command summaries point to the online version of the book on Safari Bookshelf.

Buy it now, or read it online on Safari Bookshelf.



wget

wget [options] [urls]

Perform non-interactive file downloads from the Web. wget works in the background and can be used to set up and run a download without the user having to remain logged on. wget supports HTTP, HTTPS, FTP, as well as downloads through HTTP proxies. wget uses a global startup file that you may find at /etc/wgetrc or /usr/local/etc/wgetrc. In addition, users can define their own $HOME/.wgetrc files.

Options

-a logfile, --append-output=logfile

Append output messages to logfile, instead of overwriting the contents. If logfile doesn't exist, create it.

-A acclist, --accept=acclist

Specify a comma-separated list of filename suffixes or patterns to accept.

-b, --background

Go into the background immediately after startup, writing output to the file specified with -o or to wget-log.

-B url, --base=url

Used with -F to prepend the specified URL to relative links in the input file specified with -i.

--bind-address=address

When making client TCP/IP connections, bind( ) to the specified local address, which can be specified as a hostname or IP address. Useful if your system is bound to multiple IP addresses.

-c, --continue

Continue getting a partially downloaded file. Affects the restarting of downloads from an earlier invocation of wget. Works only with FTP servers and HTTP servers that support the Range header.

--connect-timeout=seconds

Set the timeout for a connection to be established in seconds. The default is never to time out, unless a timeout is implemented by system libraries.

--cut-dirs=num

Ignore the specified number of directory components when creating the local directory structure.

-d, --debug

Turn on debugging. wget must have been compiled with debug support.

-D domainlist, --domains=domainlist

Specify a comma-separated list of domains to be followed. Does not turn on -H.

--delete-after

Delete each retrieved file from the local machine after downloading it. Useful for prefetching pages through a proxy. -k is ignored if specified with --delete-after.

--dns-cache=off

Turn off DNS-lookup caching.

--dns-timeout=seconds

Set the DNS lookup timeout to seconds. The default is to never time out.

-e command, --execute=command

Execute the specified command after the commands in .wgetrc, overriding any .wgetrc commands. Can be included multiple times, once for each command to execute.

--exclude-domains=domain-list

Specify a comma-separated list of names that are never to be followed.

-F, --force-html

When reading input from a file, force the file to be treated as an HTML file.

--follow-ftp

Follow FTP links from HTML documents. The default is to ignore FTP links.

--follow-tags=list

Specify a comma-separated list of tags to be considered, overriding the internal table that wget normally uses during a recursive retrieval.

-h, --help

Display usage information and exit.

-H, --span-hosts

Enable spanning across hosts when doing recursive retrieval.

--header=header

Add an additional header to be passed to the HTTP server. The header must include a colon (:) preceded by at least one nonblank character, and with no newline characters. Can be specified multiple times. If header is an empty string, all user-defined headers are cleared.

--html-extension

Append the suffix .html to the filenames of downloaded files where the URL does not include it (for example, an .asp file).

--http-user=user, --http-password=password

Specify the username and password on an HTTP server.

-i file, --input-file=file

Read URLs from the specified file. URLs specified on the command line are accessed before URLs in the file.

-I list, --include-directories=list

Specify a comma-separated list of directories to follow when downloading. The list elements may contain wildcards.

--ignore-length

Ignore the "Content-Length" header on the HTTP server.

--ignore-tags=list

Specify a comma-separated list of tags to be ignored for recursive retrievals.

-k, --convert-links

Convert document links after the download is complete so they work locally.

-K, --backup-converted

When converting a file, back up the original and add a .orig suffix. Affects the behavior of -N.

--keep-session-cookies

Causes --save-cookies to also save session cookies.

-l depth, --level=depths

For recursive retrievals, specify the maximum recursion depth. The default depth is 5.

-L, --relative

Follow relative links only.

--limit-rate=rate

Set the maximum download speed, The default is to specify the rate in bytes, or add a k suffix for kilobytes or m for megabytes.

--load-cookies=file

Load cookies from the specified file before the first HTTP retrieval.

-m, --mirror

Turn on options suitable for mirroring a remote site. Equivalent to -r -N -l inf --no-remove-listing.

-N, --timestamping

Turn on timestamping.

-nc, --no-clobber

Do not download a file if there is already a copy on the disk. The default is to preserve the original copy and rename successive downloads, adding .1, .2, etc. to their name. May not be specified with -N.

-nd, --no-directories

Do not create a directory hierarchy when doing recursive retrievals.

-nH, --no-host-directories

Disable creation of directories prefixed by the name of the host. The default is to include the hostname.

--no-cache

Disable server-side cache for an HTTP retrieval. The default is for caching to be on.

--no-cookies

Disable the use of cookies.

--no-glob

Turn off FTP globbing to prevent the use of wildcards for multiple file retrievals.

--no-http-keep-alive

Turn off the keep-alive feature for HTTP retrievals.

--np, --no-parent

In recursive retrievals, do not ever go up to the parent directory.

--no-remove-listing

Do not remove the temporary .listing files generated by FTP retrievals.

-nv, --non-verbose

Turn off verbose mode, but don't run completely quietly. Displays error messages and basic information.

-o logfile, --output-file=logfile

Log output messages to logfile, instead of the default standard error.

-O file, --output-documents=file

Concatenate all documents into the specified file. If the file exists, it is overwritten. Specify the file as - to write to standard output.

-p, --page-requisites

Download all files necessary to display an HTML page.

-P prefix, --directory-prefix=prefix

Set the directory prefix to the specified value.

--passive-ftp

Perform a passive FTP retrieval.

--post-data=string, --post-file=file

Use POST as the method for HTTP requests and send the specified data in the request body. Use --post-data to send string as data and --post-file to send the file contents.

--progress=type[:style]

Set the progress indicator to type. Valid types are dot and bar; the default is bar. With --progress=dot, you can also set a style. The default style is for each dot to represent 1K, with 10 dots in a cluster and 50 dots per line. Alternatives are binary, with each dot representing 8K, 16-dot clusters, and 48 dots per line; mega, for downloading very large files, with each dot representing 64K, 8 dots per cluster, and 48 dots per line; and giga, with each dot representing 1M, 8 dots per cluster, and 4 clusters per line.

--protocol-directories

Use the protocol name as part of the local filename.

--proxy-user=user, --proxy-passwd=password

Specify the username and password for authentication on a proxy server.

-q, --quiet

Run quietly; don't produce output.

-Q quota, --quota=quota

Specify download quota for automatic retrievals. The default value is in bytes; add k suffix for kilobytes, or m for megabytes.

-r, --recursive

Turn on recursive retrieving.

-R rejlist, --reject=rejlist

Specify a comma-separated list of filename suffixes or patterns to reject.

--random-wait

Set a random wait time to prevent being identified by web sites that look for patterns in time between requests so they can block access.

--read-timeout=seconds

Set the read (and write) timeout to the specified number of seconds. The default is 900 seconds.

--referer=url

Include a "Referer: url" header in an HTTP request.

--restrict-file-names=mode[,nocontrol]

Restrict the characters found in remote URLs from appearing in local filenames. The value of mode is the operating system—e.g., unix or windows (use unix for Linux). Such characters are escaped with a percent sign (%). The default is to escape characters not valid on your operating system. Appending ,nocontrol turns off escaping of control characters.

--retr-symlinks

When retrieving FTP directories recursively, follow symbolic links and retrieve the linked-to files.

-S, --server-response

Print HTTP server headers and FTP server responses.

--save-cookies=file

Save cookies in the specified file before exiting. Does not save expired cookies, and only saves session cookies if --keep-session-cookies is also specified.

--save-headers

Save the headers sent by an HTTP server to the file, preceding the contents and separated by a blank line.

--spider

Behave like a web spider, checking that pages exist but not downloading them.

--strict-comments

Turn on strict parsing of HTML comments, instead of terminating comments at the first occurrence of -->.

-t num, --tries=num

Set the number of retries to the specified value of num. Set num to 0 or inf to keep trying forever (infinitely) (default is 20 retries), unless there is a fatal error such as "connection refused."

-T seconds, --timeout=seconds

Set network timeout to the specified number of seconds. Equivalent to specifying all of --dns-timeout,--connect-timeout, and --read-timeout.

-U agent,--user-agent=agent

Specify an agent string to the HTTP server to replace the default identification of Wget/version, where version is the current wget version. This string is used in the User-Agent header field.

-v, --verbose

Turn on verbose output, printing all available data. This is the default.

-V, --version

Display version information and exit.

-w seconds, --wait=seconds

Specify the wait in seconds between retrievals. Used to lighten server load. Use the suffix m to specify the wait in minutes, h for hours, or d for days.

--waitretry=seconds

Specify the number of seconds to wait between retries if the download fails. The default in the global configuration file is to not wait.

-x, --force-directories

Create a hierarchy of directories even if one wouldn't otherwise be created.

-X list, --exclude-directories=list

Specify a comma-separated list of directories to exclude from download. List elements may contain wildcards.

-Y on|off, --proxy=on|off

Turn proxy support on or off (default is on).


More Linux resources from O'Reilly >>

Popular Topics

Browse Books & Videos

International Sites

O'Reilly China O'Reilly Germany O'Reilly Japan