Finding Messages with pick

[previous] [next] [table of contents] [index]

Let's say that you've used MH for a few months and you've got a lot of messages. Somewhere in your misc (miscellaneous) folder, buried in a message from someone (you can't remember who), is a recipe for pickled herring. (Yuck.) How can you find it? Well, you could scan the folder and show some likely messages, then read through them until you find the one with the recipe. Or you could make MH search by typing:

% pick -search herring +misc
324
That tells pick to search through every line of every message in the misc folder and show you the message numbers that have the word "herring" in them. The search matches in upper- or lowercase: it will also find "Herring," "HERRING," and so on. pick found that word in message 324. If you type show 324, voilà! (Then, if I were you, I'd type rmm...)

pick Switches

pick can do more than just -search entire messages. It will work more quickly if you only search each message header -- say, the Subject: or the From: field. pick can search for messages -to a certain user (or host). pick can find messages sent on a certain -date or all messages sent -before or -after a date. You can use several different date formats.

For example, pick -to jtq@orange would hunt for any message that contained the string jtq@orange anywhere in its To: field. The message doesn't have to be one that you sent (though you can tell pick to do that -- see the Section Combining pick Switches. As long as jtq@orange is somewhere in the To: field, pick will find the message(s) and return the message number(s). And unless you know a jtq on more than one computer, just try pick -to jtq. If a header field has a name or comment, like "John Q. Smith" or (John Smith), pick -to will match those too. So pick -to Smith might do what you need.

To find messages from zelda, use:

% pick -from zelda
12
23
Another example:
% pick -subject 'research project'
would find messages with the phrase "research project" (uppercase or lowercase letters) anywhere in its Subject: field. (When you search for a phrase with spaces or other special characters in it, put quotes around the string you're searching for.) pick can search for messages that were sent before or after a certain date. The syntax for date entry is in the order day, then month, then year. You can use special day names like today and yesterday -- also saturday, monday, and so on -- and you can give dates "n days ago." Here are some examples:

NOTE: Remember, -after searches include the date you give, but -before searches do not.

When you're looking for messages sent on a certain date, pick makes you type the date exactly as it's shown in the Date: field, in the message header (but you don't have to capitalize the month). For instance, a typical date field in a message header looks like this:

Date: Fri, 23 Dec 1994 05:51:20 PST
To find messages sent on December 23, 1994, use:
% pick -date '23 dec 1994'
(Remember: when you search for something with spaces in it, put quotes around the search pattern.) Because pick is doing a textual search, you can search for any part of the Date: field. For example, a quick-and-dirty way to find all messages sent in December 1994 is:
% pick -date 'dec 1994'
That won't match Date: fields like 23-dec-1994, of course. And that brings up another important point. With the year 2000 coming soon, many mailers have converted to four-digit years. MH converted after version 6.7.2. That adds more complication to the -date switch. Either of the next two searches would find a message sent in June 1993:
% pick -date 'jun 93' -or -date 'jun 1993'
% pick -date 'jun.*93'
The first one uses the -or switch. The second uses a UNIX regular expression to match jun, followed by any number of characters, followed by 93. That second one would also match a date with dashes between its parts, like 23-jun-93 or 23-jun-1993. For more information, see How Searches Find Messages. has more information.

If you aren't careful, regular expressions can match lines you don't expect. Unless you know exactly how each message's Date: field is written, it may be easier to use a search like the following:

% pick -after 1-dec-1994 -and -before 1-jan-1995
The sidebar Getting Picky About Date Searches has more information about pick -date.

Let's summarize. The -date switch does a textual (string) search through the messages. The -before and -after switches parse the date field and do a numeric comparison.

Passing Message Numbers with Backquotes

So far, you've seen pick output lists of message numbers, which isn't too useful. There are better things to do. You can use backquotes (`) so the UNIX shell will collect this list of messages and pass it to another command.

Here's an example. The first command gives a list of messages. The second command passes the message numbers to the scan command, which scans them:

% pick -from joed
1
2
113
227
% scan `pick -from joed`
   1  01/09 Joe Doe            Test message<<Hi!>>
   2  01/09 Joe Doe            Another test<<Well, this is anot
 113+ 01/11 Joe Doe            The latest on my project<<It's g
 227  01/13 Joe Doe            <<I can't get MH to work so I'm
If you get an error like scan: bad message list pick -from joed, you probably used the wrong kind of quotation marks. Be sure to use backquotes (`), not single quotes (') or double quotes (").

The pick -sequence switch causes problems, too. If you use -sequence on the pick command line or in your MH profile, be sure to add the -list switch, too. The -list switch forces pick to list the message numbers.

Backquotes work with any MH command -- not just scan. For instance,

% refile `pick -before -365` +old
would run pick -before -365 to get a list of all messages more than 1 year (365 days) old and then use refile to move them into the old folder.

If pick doesn't find any messages, it prints an error for you. It also outputs the illegal message number 0 (zero) for other MH commands so they'll print their own error message and quit:

% scan `pick -subject xyzxyzxyz`
pick: no messages match specification
scan: no messages match specification

Storing Message Numbers in MH Sequences

The previous section explained how to use the shell's backquoting feature to pass message numbers from pick to another MH command. There's another way to pass message numbers from pick to other MH commands -- with MH sequences. If you want to use the list of messages more than once, sequences are more efficient than backquotes.

A sequence is a named list of message numbers. This section only covers them enough for use with pick. See the Section More About Sequences.

Before you tell pick to store the message numbers in a sequence, choose the sequence name. (I usually choose the name picked because it's easy to remember.) After you run pick, MH will keep track of the message numbers in the sequence. The list of message numbers in the sequence won't change until you run pick again with the same sequence name. (If you sort or pack the folder or remove or refile messages, MH will update all the sequences automatically.)

In the following example, the first command makes a list of message numbers, as the examples with backquotes did. Because I type -sequence picked, pick stores the message numbers in a sequence named picked and prints 4 hits to tell me it found four messages. In the second command, I give the sequence name to scan, which reads the message numbers from the sequence and scans them. Finally, I use the sequence again -- this time, to remove the four messages with rmm.

% pick -from joed -sequence picked
4 hits
% scan picked
   1  01/09 Joe Doe            Test message<<Hi!>>
   2  01/09 Joe Doe            Another test<<Well, this is anot
 113+ 01/11 Joe Doe            The latest on my project<<It's g
 227  01/13 Joe Doe            <<I can't get MH to work so I'm
% rmm picked
The advantage of sequences over backquoting is that you only have to run pick once; MH remembers the message numbers.

Notice that when you use the -sequence switch, pick automatically sets its -nolist switch too. Instead of a list of message numbers, you get a summary like 4 hits. If you want the message numbers instead, type -list on the command line or put it in your MH profile.

By default, pick replaces the list of messages in an existing sequence with the new list it finds. The -nozero switch tells pick to combine the new messages with ones already in the sequence. For example, you could keep a sequence called problems and add new messages to it over time. To start the sequence, you would not use -nozero (you could use -zero if you wanted to, but that's the default):

% pick -search 'needs maintenance' -sequence problems
24 hits
Later, to add other messages to the sequence, use -nozero:
% pick -search 'needs maintenance' -nozero -sequence problems
27 hits
That would merge any new messages containing the words "needs maintenance" into the sequence with the messages it kept before. Message numbers won't be duplicated; in other words, if a message is already in the sequence, it won't appear twice after doing pick -nozero.

Storing Sequence Name in Your MH Profile

If you like having pick store the messages it finds in a sequence, you can have that happen automatically. Remember that your MH profile (usually, the file named .mh_profile in your home directory) holds default switches for MH commands, among other things. Choose a sequence name you want to use (like picked), then edit your MH profile and add this entry to it:
pick: -sequence picked
Now, whenever you use pick, it will automatically store the message numbers in the picked sequence.

But what if you want to use backquotes with pick sometimes? You'll need a list of message numbers, and the -sequence switch prevents that. The answer: use the -list switch, too; it tells pick to list the message numbers (they'll be stored in the sequence as well). You can add -list to the end of the pick: entry in your MH profile so that pick will always list the message numbers -- or you can use -list on the command line whenever you need it.

Combining pick Switches

You can pick messages based on more than one criterion. For instance, to find all messages to xavierq or to jpk, use -or:
% pick -to xavierq -or -to jpk
(Notice that you need to use -to before each addressee.) Finding all messages from perryp sent more than seven days ago needs the -and switch because you want messages that meet both of those criteria:
% pick -from perryp -and -before -7
Use -not to find messages that do not match. For instance, to find all messages that are not from you (assuming your username is joannaq), type:
% pick -not -from joannaq
If you've done much searching before (in databases, computer science courses, and so on), you probably know what operator precedence means. If you don't know, and you want to do much searching using -and or -or, it's good to get some practice first. The next example shows a typical precedence problem:
% pick -to annk -or -to ronb -and -before sunday
Does that command: The Table below lists the pick operator precedence.

Table: pick Operator Precedence

Precedence  Operator

high        -lbrace, -rbrace
  .         matching primitives (-from, -to, -search, etc.)
  .         -not
  .         -and
low         -or
If you're new to the idea of operator precedence, your best bet is to use the grouping switches -lbrace and -rbrace. Their names stand for "left brace" and "right brace." Use them to group other operators, as you'd use parentheses in algebra. Let's make the example above unambiguous:
% pick -lbrace -to annk -or -to ronb -rbrace -and -before sunday
Now it says you want messages sent to annk or to ronb before Sunday. With real braces (which don't work with pick, unfortunately), that would look like:
pick { to annk or to ronb } before sunday
The braces tell pick exactly which two terms you want the -or applied to.

And remember -- you can abbreviate those switches:

% pick -lb -t annk -o -t ronb -rb -an -b sunday

picking Miscellaneous Fields

You've seen how to search for messages by Subject:, To:, and From:. (You can also search the cc: field with -cc.) You can search for any other field -- like Reply-To:, Return-Path:, and so on. Use the -- field-name switch (with two dashes before the switch name). For example, to search for any message which had the field Sender: xandra@xyz.com, you would type:
% pick --sender xandra@xyz.com
repl -annotate and forw -annotate add Replied: and Forwarded: fields to message headers. So you could search for all messages which had been forwarded by typing:
pick --forwarded .
The dot (.) means "match any character." You need the dot because pick requires something to search for. If the field might be empty, replace the single dot with '.*', an expression in single quotes that matches zero or more characters. The Section How Searches Find Messages has more info.

To search for messages that have not been forwarded:

pick -not --forwarded .
Here's another place I've used this. I was one of several consultants in a computer center. We got a lot of questions by email. When we answered a message, we put a field in the reply header with the initials of the consultant who answered it (we all answered from the consult account). For example, the header of a message I sent would start like this:
From: consult
To: auser
X-By: jdp
cc:
(You can make up arbitrary fields if the name starts with X-. We named ours X-by:.) To search for the messages that I (jdp) sent, we typed:
% pick --x-by jdp
You can set up your message headers this way before you send the messages or after you receive them. For help, see the parts of this book about customizing message headers -- like the Sections Draft Message Template Files and Annotating Headers with anno.

How Searches Find Messages

When you use pick switches that search text -- like -subject, -to, -search, and others -- you may find messages you didn't expect. This section explains why and explains how to make your search more accurate. It also shows examples of searching with regular expressions, which let you match ranges of text without typing all the text exactly. (This section doesn't apply to switches like -before and -after that search for messages by date.)

Messages You Don't Want

pick searches all of a field. For example, if you type:

% pick -to ed
it will find all of these messages:
To: ed
To: ed@someplace.xxx
To: FRATED@UCI
To: kelly@bigcorp.com, mongo@medfly.com
because each of those fields has an uppercase or lowercase "ed" somewhere in it.

After you know pick works that way, you can take advantage of it. For example, to find all messages from any user at temptron.com, you could use a search like:

% pick -from temptron.com
assuming that all of their mail has temptron.com in its From: fields.

Now, back to the first example. What if you want to find mail to Ed, not to frated or mongo@medfly.com? Give Ed's address as exactly as you can. For example:

% pick -to ed@someplace.xxx
would do the trick, assuming that Ed's address looks exactly like that in the To: field. In other words, if Ed's address looked like this in some messages:
ed%somewhere@someplace.xxx
then you'd have to type a search line like this:
% pick -to ed@someplace.xxx -or -to ed%somewhere@someplace.xxx
As explained in the Section Using Regular Expressions, you could also use an expression to match any number of characters between the parts of the address:
% pick -to "ed.*@someplace.xxx"

Uppercase vs. Lowercase

If you type your search in lowercase letters, pick will match either lowercase or uppercase letters. As an example, if you want to find messages with the name "Sam" anywhere, the search:

% pick -search sam
would also find messages that have the word "flotsam" or a field such as To: kenny@samron.com.

If you type an uppercase letter, pick will match it exactly. So a much better search (that still finds the words "Samuel" and "Samantha", by the way) is:

% pick -search Sam

Watch Out for the Shell

"Aha," you think, "I could find Sam by typing his whole name, Sam Spade. Then `Samuel' or `Samantha' wouldn't match." Try it, if you'd like:

% pick -search Sam Spade
pick: bad message list Spade
You'll get an error. Why? Because the UNIX shell splits the words at the space, and pick thinks that the second word is a list of messages (like a sequence). The answer: put single quotes around the string you're searching for. Be sure to use two single quotes (''), not the backquotes (``), as shown in the following example:
% pick -search 'Sam Spade'
That won't match messages with "Sam" at the end of one line and "Spade" at the start of the following line, but it's progress....

Using Regular Expressions

pick can do more sophisticated searches with UNIX regular expressions. A regular expression uses special pattern-matching characters like these:

[ ] . *
to find arbitrary combinations of text.

pick uses regular expressions like the ones UNIX utilities such as grep(1) or editors like vi(1) and ed(1) do. Here are two quick examples:

  1. You can write a regular expression to find a message with a subject that has the word "report" followed by the word "1994", any place after it on the line. Some of the subjects we want look like this:
    Subject: status report for May, 1994
    Subject: Report on the month of August, 1994
    
    The pick command you'd type is:
    % pick -subject 'report.*1994'
    
    The dot followed by an asterisk (.*) means "zero or more of any character."
  2. You'd like to find messages with information about your company's product with model numbers of AB1234x, AB1234y, or AB1234z. Without regular expressions, you'd have to type:
    % pick -search AB1234x -or -search AB1234y -or -search AB1234z
    
    But with a regular expression that matches the three different last characters in those numbers, you could type:
    % pick -search 'AB1234[xyz]'
    
    pick doesn't support the pattern syntax [l-r] (with a dash inside the square brackets); each letter to be matched must be included within the brackets.

Even More Details...

This section, adapted from the MH 6.7 pick(1) manual page, has more precise information about pick searches. If you don't want this much detail, feel free to skip ahead.

pick searches messages within a folder for the specified contents and then identifies those messages. Two types of search primitives are available: pattern-matching and date-constraint operations.

A modified grep(1) is used to perform the matching, so the full regular expression (see ed(1)) facility is available for the search pattern. With -search, the pattern is used directly; with the others, the grep pattern constructed is:

field[ \t]*:.*pattern
(where \t is a tab character). This means that the pattern specified for a -search will be found everywhere in the message, including the header and the body, while the other pattern-matching requests are limited to the single specified field. The expression:
 --field pattern
is a shorthand for specifying:
-search 'field[ \t]*:.*pattern'
Pattern matching is performed on a per-line basis. Within the header of the message, each field is treated as one long line, but in the body, each line is separate. Lowercase letters in the search pattern will match either lowercase or uppercase in the message, while uppercase will match only uppercase.

Independent of any pattern-matching operations requested, the switches -after date or -before date may also be used to introduce date/time constraints on all of the messages. By default, the Date: field is consulted, but if another date yielding field (such as BB-Posted: or Delivery-Date:) should be used, the -datefield field switch may be used. pick will actually parse the date fields in each of the messages specified (unlike the -date switch which does a pattern-matching operation) and compare them to the date/time specified by use of the -after and -before switches. If -after is given, then only those messages whose Date: field value is chronologically after the date specified will be considered. The -before switch specifies the complementary action. (One of this book's reviewers points out that MH doesn't parse European timezones -- EET DST, EETDST, CET, and so on. They default to GMT.)

Both the -after and -before switches take legal RFC 822-style date specifications as arguments. pick will default certain missing fields so that the entire date need not be specified. These fields are (in order of defaulting): time zone, time and time zone, date, and date and time zone. All defaults are taken from the current date, time, and time zone. In addition to RFC 822-style dates, pick will also recognize any of the days of the week (sunday, monday, and so on) and the special dates today, yesterday, and tomorrow. All days of the week are judged to refer to a day in the past (e.g., telling pick saturday on a Tuesday means last Saturday not this Saturday). Finally, in addition to these special specifications, pick will also honor a specification of the form -dd, which means "dd days ago."

Searching a Message Range or Sequence

If your mail folders are full of messages, each pick you run can take quite a bit of time. I like to limit my search to a range of messages.

For example, if I know the message I want is in the last 200 messages, I can type:

% pick -to auser last:200
Without the last:200, pick would search all 4000 messages in the folder. This way is much faster.

The string last:200 is an example of the predefined MH message number ranges. You can also use message ranges, like pick -to auser 25-150, to search only message numbers 25 through 150.

Or you can define a sequence of messages and then use that sequence as a starting place for other sequences. For example, let's say that you manage a group of people. You have a folder with 600 messages from your employees. You want to find the 20 or 30 messages with a subject of "status report" and split those messages into folders by employee name. The inefficient way to do that would be to search all 600 messages, for each of your ten employees, like this:

% refile `pick -from pwb -and -subj "status report"` +paul_b
% refile `pick -from jsp -and -subj "status report"` +joan_p
% refile `pick -from dac -and -subj "status report"` +don_c
% ...
pick would have to search almost 600 messages each time (that's slow). Instead, tell pick to search all 600 messages just once to find all the status reports and store those 20 or 30 message numbers in a sequence. Then, to find an employee's individual status report, have pick search the sequence of 20 or 30 messages (that's fast). Notice that we only use the -sequence switch once to store the sequence:
% pick -subject "status report" -sequence picked
% refile `pick picked -from pwb` +paul_b
% refile `pick picked -from jsp` +joan_p
% ...
and so on. (This example assumes that the MH profile does not have pick: -sequence picked in it. If it did, each of those pick commands would overwrite the previous contents; we'd need to use a different sequence name instead.)

If you'd like a look back at how pick worked in early MH versions, read the sidebar Prehistoric pick-ing.

Searching More Than One Folder

If you don't know what folder a message is in, use a shell loop to run pick on several folders. You can use backquotes and the folders -fast command to give the loop a list of all your top-level folders. Adding -recurse to folders will search your subfolders, too -- but it can also be a lot slower. Use the -sequence switch to store a list of the messages pick finds in a sequence in each folder. Then you can go to folders and check the messages in that sequence.

The Example below shows two loops. The first runs pick on all my top-level folders and stores messages it finds in each folder's picked sequence. Let's say I notice three likely folders where pick found matches ("hits"). In the second loop, I run scan on those three folders.

Example: Searching all top-level folders
C shell:

% foreach f (`folders -f`)
? echo "Checking $f"
? pick pick switches +$f
? end
Checking apple
pick: no messages match ...
    ...
Checking zebra
12 hits
% foreach f (chk memos zebra)
? scan picked +$f
? end
...Messages found in chk folder...
...Messages found in memos folder...
...Messages found in zebra folder...
Bourne and Korn shells:
$ for f in `folders -f`
> do echo "Checking $f"
> pick pick switches +$f
> done
Checking apple
pick: no messages match ...
    ...
Checking zebra
12 hits
$ for f in chk memos zebra
> do scan picked +$f
> done
...Messages found in chk folder...
...Messages found in memos folder...
...Messages found in zebra folder...
Instead of using a separate loop to scan, you can use the shell's && operator to run scan only when pick returns a "true" exit status in a folder. Change the pick line in the first loop of the previous Example to look like this:
pick pick switches +$f && scan picked
Another great idea (thanks to Bill Wohler) is to link all the matching messages into a temporary folder. When you're finished with the linked versions, remove the temporary folder. Just change the pick line to be:
pick pick switches +$f && refile -link picked +temp
You can get fancier, storing folder names with a match in a shell variable and so on. But that's probably the place to write a little shell script.

Easier Searches with a "Link Folder"

Running pick across a bunch of folders, as you saw in the Section Searching More Than One Folder, can still be inconvenient. You might want to operate on messages from several folders at the same time. Or, you might just be sick of typing shell loops. The Section A Folder Full of Links has an idea: a folder full of links to messages in other folders. When you want to search for messages and you know what folder to look in, search that folder. Otherwise, search the folder with links to many other folders -- it'll take more time because there are more messages to search, but it sure is easier.

[Table of Contents] [Index] [Previous: Folders] [Next: More About Sequences]


Last change $Date: 1996/06/06 15:10:15 $

This file is from the third edition of the book MH & xmh: Email for Users & Programmers, ISBN 1-56592-093-7, by Jerry Peek. Copyright © 1991, 1992, 1995 by O'Reilly & Associates, Inc. This file is freely-available; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation. For more information, see the file copying.htm.

Suggestions are welcome: Jerry Peek <jpeek@jpeek.com>