bash Tricks From the Developers of the O'Reilly Network
 | ||
 | Email weblog link | |
 | Blog this |
Tony Stubblebine
Apr. 21, 2005 06:09 PM
Permalink
Every week the developers of the O'Reilly Network (that's me, three developers, and two admins) have a status meeting to check in with our key managers, decide or rearrange priorities, and work through problems. This is a dream meeting for managers, questions are answered and plans are laid.
It's fair to say that status meetings aren't a developer's dream. After several years of weekly meetings, ours were feeling stale. So we agreed to end the meetings with a round of tips and tricks. First up, bash tricks.
At the end of our meeting, the managers bailed and we stuck around to geek out over the tricks that make our work easier. Everyone had something to contribute. I can't recommend this enough! I've learned a lot from books like Unix Power Tools. But by sharing directly with your coworkers you get advice that's targeted directly to the work you do.
Here were our gems, the most useful tips that weren't already common knowledge among the developers.
pushd/popd
Bash will keep a history of the directories you visit, you just have to ask.
Bash stores the history in a stack and uses the commands pushd
and popd to manage the stack.
pushd dir - move the current directory onto the stack and change to
the dir directory.
popd - pops the top directory off of the stack and moves you into it.
We're opening files all over the file system, internal code, vendor code, templates, configuration files, logs. Because of this we like the ability to take a detour on the file system and still navigate back to our working directory of the day. I think these commands are so useful that I alias'd them in my .bashrc
alias cd="pushd"
alias bd="popd"
Now the cd command manages the stack for me as well as changing directories. Aliasing popd to bd is an easy to remember and easy to type way to move back up the stack, think "change dir" and "back dir".
History
Bash keeps a history of the commands you've run. My group was already comfortable with the up and down arrows to navigate the history, !! to repeat the last command, and !foo to repeat the last command starting with foo.
Our newest admin had a better way, CTRL-R. That does command auto completion. Repeatedly pressing CTRL-R lets you tab through matching commands.
Home/End
CTRL-A takes you to the beginning of the line and CTRL-E takes you to the end of the line. This is probably basic shell knowledge, but I'm probably (hopefully) not the only person who didn't know it.
For Loops
We've got a cluster of machines that we'll sometimes need to loop through. Here's an example from our admins that checks uptime across our cluster.
$ for s in `cat server.list`; do ssh $s uptime; done;
Working with the Previous Command
Sometimes you want to run several commands on the same file, like run ls before deciding if that's the file you want to edit.
ls -l /long/path/to/file.txt
vi /long/path/to/file.txt
Bash provides a shortcut (!$) that holds the last word from the previous command. So in the above you could just write vi !$.
If your last command had a typo you can fix the command and rerun it with this construct, ^foo^bar. That replaces the first occurrence of foo in your command with bar.
Bonus Tip: use Awk
Our admins seem to think awk is pretty useful. And my boss thought it was so useful, he wrote a book on it. I can't keep any of the awk syntax in my head beyond printing out a column from a file.
The normal column delimiter is whitespace. So if you wanted to print out the seventh column in an Apache access log (that's the request url in my logs) you could write:
cat access_log | awk '{print $7}'
You can change the delimiter with -F. So if you wanted to list all the users on your system, you could pull them out of /etc/passwd with:
$ cat /etc/passwd | awk -F: '{print $1}'
The /etc/passwd delimiter is :, which I've indicated
to awk with -F:.
We're doing vim tricks next.
Tony Stubblebine is an Internet consultant and author of Regular Expression Pocket Reference.
Showing messages 1 through 17 of 17.
-
No more worrying about cases
2005-05-22 19:16:37 dustbo [View]
-
directory stack
2005-04-28 09:36:36 rarue [View]
Long ago having aliased "pu" and "po" to pushd and popd, I've moved on to showing my current directory stack with
alias d='dirs -v'
and rolling through the stack with
alias r='pushd +1'
and single numerical values aliased to go to that directory that comes from the dir listing, eg.
alias 3='pushd +3 > /dev/null ; dirs -v'
Another single letter alias I like is "p", which I have used for displaying and setting my PATH variable, but the code is lengthy so I won't burden you with how
p - 4
p --
p ++ /some/dir
does what you want.
-
Command substitution
2005-04-28 07:45:02 pkailasa [View]
$ for s in `cat server.list`; do ssh $s uptime; done;
Command substution is also done using$(command)notation, which I prefer to the backquotes. It allows commands to be nested (backquotes allow that too, but the inner quotes must be escaped using backslashes, which gets messy.
For example:
$ for s in $(cat server.list); do echo "$s: $(ssh $s uptime)"; done;
or:
# get the uptime for just the first server
$ echo "$(date): $(ssh $(head -1 server.list) uptime)"
-
basename
2005-04-24 17:59:30 bingalls [View]
basename also supports extensions. Try this, to rename *.xml to *.wsdl (or *.cxx to *.cpp, etc)
for i in *.xml;do;mv $i `basename $i .xml`.wsdl;done
I tried this in zsh, so bash may complain that I have an extra ;.
Be sure to upgrade to the latest bash, which has largely caught up with zsh features like **/* and programmable completion.
-
should be !$
2005-04-22 10:38:20 pga-eti [View]
Instead of $!, use !$, it works much better. :)
$ echo asdf
asdf
$ echo !$
echo asdf
asdf
$ echo $!
$
So $! is an empty variable, while !$ brings back the last argument from the last command. -
should be !$
2005-04-22 10:43:07 tonystubblebine | [View]
So true. I updated the entry with your correction. Thanks.
-
ugu.com
2005-04-22 06:12:31 msporleder [View]
Get UNIX tips in your email every day, or enter some new ones to get emailed out to me and whoever else does this:
http://www.ugu.com/sui/ugu/show?I=tip.today
-
vi key bindings
2005-04-22 04:28:19 mrchucho@yahoo.com [View]
set -o vi
and you can navigate through the line using vi-style key bindings (e.g.g $ end of line, 0 start of line, etc.) Similarly, "ESC k" goes "up" in history and, "j" will go back down. "ESC /<string>" will search history for a string. And so forth! -
vi key bindings
2005-04-22 08:23:13 tonystubblebine | [View]
This is a good one! Our other development team here (the people bringing you SafariU and the data that shows up on O'Reilly Radar) swear by it.
I'm a vi user which is why I didn't know the home/end keymappings for Bash. Probably a good idea for any vi users to make this switch.
-
xargs and useless use of cat
2005-04-22 02:14:07 aristotle [View]
<blockquote>for s in `cat server.list`; do ssh $s uptime; done</blockquote>
Do investigatexargs(1). The above example is easier written as
xargs -i ssh {} uptime < server.list
<blockquote>cat access_log | awk '{print $7}'</blockquote>
That’s a useless use of
catthere. You can just say
awk '{print $7}' access_log
Some people say they prefer
cat(1)because that way they can expand or change the pipe easier. In that case, you can use a little-known feature of redirection operators: you can put them anywhere you want, ie
< access_log awk '{print $7}'
-
More key bindings and tricks
2005-04-22 00:40:26 FBoender [View]
Bash will keep a history of the directories you visit, you just have to ask.
You can also always go back to the previous directory you were in by typing cd - without the need to pushd the current directory. Using it more than once cycles between the current and previous directory.
CTRL-A takes you to the beginning of the line and CTRL-E takes you to the end of the line. This is probably basic shell knowledge,
I think it's actually common readline/emacs knowledge, and it works in much more programs than just Bash or a terminal. For instance, you can enable them in Gnome applications by adding the line
gtk-key-theme-name = "Emacs" to the ~/.gtkrc-2.0 file.
Other handy key bindings you can use are:
- ctrl-u : Cut everything on the current line before the cursor.
- ctrl-y : 'Yank' (paste) text that was cut using ctrl-u.
- ctrl-w : Delete the word on the left of the cursor
There's so much usefull knowledge hidden in Bash that, if you spend any time at the command line, you should really get yourself aquinted with. It saves incredible ammounts of time.
Take for example something I wanted to do yesterday. I wanted to now the number of hits on a certain website. I could have installed a tool to parse the Apache access.log, but this was much easier:
$ cat access.log | cut -d"[" -f2 | cut -d"]" -f1 | cut -d"/" -f2 | uniq -c
28905 Mar
16554 Apr
Takes no more than a couple of seconds to write, but saves so much time.
Try reading through the Bash man page. It's huge, but think of all the stuff you'll learn! Or read some online Bash scripting tutorials. Everything from gathering statistics from files to creating thumbnails of images (From the top of my head: for A in *; do convert $A -resize 140x140 th_$A; done) becomes a cinch.
-
Excellent
2005-04-21 22:11:44 Marc Hedlund |
[View]
Fantastic post. Great idea for improving meetings, and great tips.
-
This week's Useless Use of Cat Award Goes to...
2005-04-21 22:03:42 Randal L. Schwartz |
[View]
Hey, I like your useless use of cat... perhaps you should read why they're useless. -
This week's Useless Use of Cat Award Goes to...
2005-04-22 08:28:58 tonystubblebine | [View]
I'm honored! Thanks Randal =)
-
Last argument
2005-04-21 19:26:57 Andy Lester |
[View]
You can also use Esc-period and get the last parm of the previous line. You can repeatedly use Esc-period to scroll back through time with them. That turns out to be even better than $! because you can edit it once it shows up on your command line.
| Showing messages 1 through 17 of 17. |
Return to weblogs.oreilly.com.
Weblog authors are solely responsible for the content and accuracy of their weblogs, including opinions they express, and O'Reilly Media, Inc., disclaims any and all liabililty for that content, its accuracy, and opinions it may contain.
This work is licensed under a
Creative Commons License.
create a file called .inputrc in your home directory and put this line in it:
set completion-ignore-case on
Now bash tab-completion won't worry about case in filenames. Thus 'cd sit[tab]' would complete to 'cd Sites/'