Storing Messages

[previous] [next] [table of contents] [index]

The first part of this section is mostly for people who've come to MH from another MUA, an MUA that stores all of a folder's messages in a single file. The second part discusses ways to save space when you store MH folders. The third part covers decoding and storage of MIME mail.

Copying a Message to a File

As explained in the Chapter Key Parts of the UNIX Filesystem, MH stores each message in a separate file. So, unlike most other email systems, MH doesn't need a command that saves a message to a file. (There are two exceptions. The mhn -store command decodes and stores MIME message bodies. The MH msh command simulates non-MH MUAs.)

One easy way to get the contents of a single message is by redirecting the output of show to a file (the shell's > operator) or with a pipe (|). But the plain show command formats a message for viewing on your screen. If you add its switches -noshowproc -noheader, then show will output the message without formatting.

You can also get the pathname of any MH message within your MH directory and give it to a UNIX command. Concatenate three things: the pathname of the MH directory, the folder name, and the message number. Let's say you wanted to transfer a copy of message 25 from your data folder to a personal computer, using kermit. Your MH directory is /home/marty/Mail. The command you'd type to send that message is:

kermit -s /home/marty/Mail/data/25

Of course, you can use the same message pathname to copy the message to another file (with cp), make a link to it (ln and ln -s), edit the message (with vi, emacs, etc.), and use any other UNIX command that uses a file pathname. If you edit a message file that MH will be handling later, remember to keep the correct header and body format. For example, I edit long disk-eating messages that I want to keep parts of; I replace long deleted sections with ...deleted by Jerry....

You can also use the mhpath command, which gives you message and folder pathnames.

Finally, you may want to save just the body of a message -- without the header. One easy way is by using a simple sed script that prints all lines after the first blank line:

% sed '1,/^$/d' /home/eloisa/Mail/inbox/25

You can replace the file pathname with a backquoted mhpath expression. Or, instead of using a file pathname, you can pipe the output of show to that sed command; check to see whether your show command might reformat the message body (by wrapping long lines and so on). Finally, if you've set show to format messages with mhl, you can get just the unformatted message by using the mhl.body format file:

% show -form mhl.body

Saving Filesystem Space

UNIX filesystems store files in fixed-size chunks of disk space called blocks. MH stores each message in a separate file. If a message is smaller than a block, the leftover space in that block is wasted. When you store a lot of MH messages, there's probably a lot of unused space in the blocks that make up your MH folder tree. If you're short of disk space, you'll probably want to use your disk blocks more efficiently.

Other MUAs store many messages in the same file: a single file holds an entire mail folder. So, for example, a folder with 50 messages might fit into 32 blocks: 31 of them completely full, with wasted space in the last block. MH can store messages that way, too. The Section Pack Messages into a File explains how to "pack" messages into a single file. Packing a big folder that you don't use much can save a lot of room. Once you've packed a folder, you can handle the messages with msh and its MH commands scan, show, rmm, etc. You can also use the command scan -file to see what's in the packed file.

I like to put my packed files in my MH directory, at the same level as the mail folders, and name the packed file foldername.packf. If you're archiving the same folder periodically, add a date too. For example, to pack the folder personal into a file, then remove the messages in the folder:

% packf -file /u/jerry/Mail/personal.9501.packf +personal
% rmm all

(Remember that the default rmm command doesn't remove the message files, just renames them. You won't save disk space until the "removed" messages are actually deleted. See the Section Removing and Recovering Messages.)

To get messages out of a "packed" file, use the command:

% inc -file packedfile

The messages will be read into your inbox unless you give a folder name (with a +, as usual). The messages will not be removed from the packed file unless you use inc's -truncate switch.

Compressing the file with a utility like compress or gzip saves even more disk space. You'll have to uncompress or gunzip the file to use it, though.

Using packf and rmm to pack a folder has some disadvantages:

If you have links to messages in the folder, they'll be broken. The other links will still use disk space. If you want to check for other links, use a command like:
```
find /path/to/folder -type f -links +1 -print
```
Access permissions, modification time, and other UNIX file attributes won't be preserved.

To preserve links, modification times, and other attributes of the message files, try tar(1) and its option -l (lowercase letter "L": complain about missing links). Be sure to use compress(1) or gzip(1) if you save tar output to a file; this will squeeze out the disk-space-eating NUL characters that tar adds to archive files.

The Sections Cleaning out Folders and Archiving Folders have more space-saving examples.

Decoding and Storing MIME Messages

As the Section Copying a Message to a File explains, it's easy to store the content of non-MIME messages in a file: just copy the body, or the header and body, to the file. But many MIME messages are encoded as quoted-printable or base64, or have multiple parts. So, saving MIME message bodies directly in a file isn't always what you want. This section is about mhn -store. The mimecat script in Section Decoding Messages with mimecat extracts and decodes message parts; it's also worth a look.

The mhn -store switch decodes MIME contents. The decoded content is sent to a file or piped to a program. If no decoding is needed -- for instance, if the content has a 7bit or 8bit encoding -- mhn -store simply stores the content.

The Section Partial Messages showed mhn -store combining partial messages into a new message. Here are more examples:

% mhn -store
storing msg 10 using command (cd mime-storage; gifconvert -jpeg > 10.jpg)


% mhn -store 5
storing message 5 as file mime-storage/5.txt
% mhn -store -part 1 16
storing message 16 part 1 as file mime-storage/16.1.ps

The first example stored a GIF-format message content into a JPEGformat file by running a command named gifconvert. (gifconvert is an imaginary command; I made up the name. mhn can run any command you tell it to.) The second example stored a content into the file named 5.txt. Compare that to the third example, where the part number was added to the filename.

The examples used a subdirectory named mime-storage. How did mhn do that?

To automate the process of storing contents, mhn has to make a lot of decisions. They're summarized below; the Section Configuring mhn has details. If this seems complicated, read through the summary once or twice and then experiment. Once you've thought through what needs doing, this twisty maze of steps should start to make more sense.

What's in the Profile Entry

As with most of the other things mhn does, its -store switch uses a profile entry. The entry tells mhn how to store the particular content. The entry can have two parts:

A filename template with escapes (variables) like %m (the message number), %P (the part number with a leading dot), and %s (the content subtype). For example, you might be storing part 3.2 of message 64, an image/gif content. If the template is %m%P.%s, mhn would name the file 64.3.2.gif. The Section Storing Content: mhn-storage and mhn-store- has the details and explains how to change this.
An optional UNIX command to process the decoded content before it's stored. For instance, you might be transferring a bunch of audio/basic messages to your Macintosh, and you'd like to convert the audio to AIFF format before storing it. Your profile entry could run the sox sound-converter program before storing the content.
If there's no UNIX processing command, mhn stores the decoded content as-is.

Automatic Filename Selection: -auto

There's one other way that mhn can choose a filename. If you give the -auto switch (on the mhn command line in your MH profile) and if the message has a filename parameter on its Content-type: field, then the filename template is not used. In that case, the filename in the message is used. For example, if you're storing a message with an image/jpeg content, and the message has this field:

Content-type: image/gif; name=al_gore.jpg

If you store the message with the command mhn -store -auto, the content will be stored in al_gore.jpg.

Here are three more notes about -auto:

For security reasons, if the filename in the content starts with a slash (/), mhn won't use it.
It's a good idea not to put -auto in your MH profile. Instead, use mhn -list -verbose to check for a name= parameter with a filename you wouldn't want to have overwritten. If there's no problem, use the -auto flag when you run mhn -store.
If you've added -auto to your MH profile, you can override it (and use the filename template in your profile) by giving -noauto on the command line.

Choosing a Storage Directory

The two sections above explained how mhn picks a filename. mhn also needs to know what directory to use. If the profile entry uses an absolute pathname (a pathname that starts with a / character), mhn will store the file in that directory and filename. Otherwise, if you've put a mhn-storage: entry in your MH profile, mhn will store the file in the directory you name there. With no other instructions, mhn defaults to the current directory.

In most cases, mhn -store will overwrite (replace) a existing file. This isn't true when it's adding a message to a folder or building a new message from partial messages. It also may not be true when mhn uses a separate UNIX command to store a content -- in that case, the UNIX command decides whether to overwrite the file.

Choosing a Profile Entry

How does mhn choose a profile entry? The entries are named for the particular content type (and, optionally, subtype) that you're storing. mhn uses the following steps to store a particular content:

If there's a profile entry mhn-store-type/subtype:, mhn uses that.
Otherwise, mhn looks for mhn-store-type:.
Otherwise, if the content is application/octet-stream with the parameter type=tar, mhn will convert the input (that is, undo the base64 encoding). (The MH viamail program works like this.) Then:
- If the message has a parameter ; conversions=compress or ; x-conversions=compress, mhn will pipe the output through uncompress | tar xvpf -.
- Otherwise, mhn will pipe the output through tar xvpf -.
The processed output will be stored in the filename given by the name= parameter.
If the content is a message, mhn will create a new message in the current folder. (To be exact, it will use the formatting string +, which stands for the current folder.)
Otherwise, mhn will store the content in a file named by the formatting string %m%P.%s.

[Table of Contents] [Index] [Previous: More About Sequences] [Next: Sorting Messages: sortm]

Last change $Date: 1996/06/06 15:14:26 $

This file is from the third edition of the book MH & xmh: Email for Users & Programmers, ISBN 1-56592-093-7, by Jerry Peek. Copyright © 1991, 1992, 1995 by O'Reilly & Associates, Inc. This file is freely-available; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation. For more information, see the file copying.htm.

Suggestions are welcome: Jerry Peek <jpeek@jpeek.com>