The Mail Transfer AgentBy Philip Hazel
0-596-00098-7, Order Number: 0987
632 pages, $44.95
Receiving and Delivering Messages
Coordination Between Processes
How Exim Is Configured
How Exim Delivers Messages
Local and Remote Addresses
Processing an Address
A Simple Example
Complications While Directing and Routing
Complications During Delivery
Complications After Delivery
Use of Transports by Directors and Routers
In the previous chapter, the job of an MTA is described in general terms. In this chapter, we explain how Exim is organized to do this job, and the overall way in which it operates. Then in the next chapter, we cover the basics of Exim administration before launching into more details about the configuration.
Exim is designed for use on a network where most messages can be delivered at the first attempt. This is true for most of the time over a large part of the Internet. Measurements taken in the author's environment (a British university) indicate that well over 90 percent of messages are delivered almost immediately under normal conditions. This means that there is no need for an elaborate centralized queuing mechanism through which all messages pass. When a message arrives, an immediate delivery attempt is likely to be successful; only for a small number of messages is it necessary to implement a holding and retrying mechanism.
Therefore, although it is possible to configure Exim otherwise, the normal action is to try an immediate delivery as soon as a message has been received. In many cases this is successful, and nothing more is needed to process the message. Nevertheless, some precautions must be taken to avoid system overload in times of stress. For example, if the system load rises above some threshold, or if there are a large number of simultaneous incoming SMTP connections, immediate delivery may be temporarily disabled. In these events, incoming messages wait on Exim's queue and are delivered later.
All operations are performed by a single Exim binary, which operates in different ways, depending on the arguments with which it is called. Although receiving and delivering messages are treated as entirely separate operations, the code for determining how to deliver to a specific address is needed in both cases, because during message reception, addresses are verified by checking whether it would be possible to deliver to them. For example, Exim verifies a remote sender address by looking up the domain in the DNS in exactly the same way as when setting up a delivery to that address.
The word queue is used for the set of messages that Exim has under its control at any one time, because this word is common in the context of mail transfer. However, Exim's queue is normally treated as a collection of messages with no implied ordering, more like a ''pool'' than a ''queue.'' Furthermore, Exim does not maintain separate queues for different domains or different remote hosts.
There is just a single collection of messages awaiting delivery, each of which may have several recipients. You can list the messages on the queue by running the command:
assuming that your path is set up to contain the directory where the Exim binary is located. Messages that are not delivered immediately on arrival are picked up later by queue runner processes that scan the entire queue and start a delivery process for each message in turn. A queue runner process waits for each delivery process to complete before starting the next one.
Receiving and Delivering Messages
Message reception and message delivery are two entirely separate operations in Exim, and their only connection is that Exim normally tries to deliver a message as soon as it has received it. Receiving a message consists of writing it to local spool files (''putting it on the queue'') and checking that the files have been successfully written before acknowledging reception to the sending host or local process. There is only one copy of each message, however many recipients it has, and the collection of spool files is the queue; there are no additional files or in-memory lists of messages.
A delivery operation gets all its data from the spool files. Each attempt at delivering a message processes every undelivered recipient address afresh. Exim does not normally retain previous alias, forwarding, or mailing list expansions from one delivery attempt to another.
There is, however, one exception to this: if the one_time option is set for a mailing list, the list's addresses are added to the original list of recipients at the first delivery attempt, and no re-expansion occurs at subsequent attempts.
Parallelism is obtained by the use of multiple processes, but one important aspect of Exim's design is that there is no central process that has overall responsibility for coordinating Exim's actions, and therefore there is no concept of starting or stopping Exim as a whole. Exim processes can be started at any time by other processes; for example, user agents are always able to start Exim processes in order to send messages. Such processes perform a single task and then exit. Most processes are therefore short-lived, but Exim does make use of long-running daemon processes for two purposes:
To listen on the SMTP port for incoming TCP/IP connections. On receiving such a connection, the listener forks a new process to deal with it. An upper limit to the number of simultaneously active reception processes can be set. When the limit is reached, additional SMTP connections are refused.
To start up queue runner processes at fixed intervals. These scan the pool of waiting messages (by default in an arbitrary order) and initiate fresh delivery attempts. A message may be on the queue because a previous delivery attempt failed, or because no delivery attempt was initiated when the message was received. Each delivery attempt processes a single message and runs in its own process, and the queue runner waits for it to complete before moving on to the next message. A limit may be set for the number of simultaneously active queue runner processes run by a daemon.
A single daemon process can be used to perform both these functions, and this is the most common configuration. However, it is possible to run Exim without using a daemon at all; inetd can be used to accept incoming SMTP calls and start up an Exim process for each one, and queue runner processes can be started by cron or some other means. However, in these cases Exim has no control over how many such processes are run, so if you are worried about system overload, you must control the number of processes yourself.
xinetd (www.xinetd.org) is a replacement for inetd that includes additional control facilities.
Coordination Between Processes
Processes for receiving and delivering messages are for the most part entirely independent. The small amount of coordination that is needed is achieved by sharing files. Minimizing synchronization and serialization requirements between processes helps Exim to scale well. Apart from the messages themselves, the shared data consists of a number of files containing ''hints'' about mail delivery. For example, if a remote host cannot be contacted, the time of the failure and the suggested next time to try that host are recorded. Any delivery process that has a message for that host will read the hint and refrain from trying the delivery if the retry time has not been reached. This does not affect delivery of the same message to other hosts when there is more than one recipient address.
Because the coordinating data is treated as a collection of hints, it is not a major disaster if any or all of it is lost; there may be a period of less optimal mail delivery, but that is all. Consequently, the code that maintains the hints can be quite simple because it does not have to be made robust against unusual circumstances.
How Exim Is Configured
Configuration information, supplied by the administrator, is used at two different times: one configuration file is used when building the Exim binary, and another is read whenever the binary is run. Most options can be specified in only one of these files; that is, they either control how the binary is built, or they modify its behavior at runtime, but there are a few build-time options that set defaults for runtime behavior. The sources of Exim's configuration information are shown in Figure 3-1.
Figure 3-1. Exim configuration
The build-time options are of three kinds:
Those that specify the inclusion of optional code; for example, to support specific database lookups such as LDAP, or to support IPv6.
Those that specify fixed values that cannot be changed at runtime; for example, the mode of message files in Exim's spool directory.
Those that specify default values for certain runtime options; for example, the location of Exim's log files.
The process of building Exim from source is described in detail in Chapter 22, "Building and Installing Exim". Here, we consider the runtime configuration. This is controlled by a single text file, often called something like /etc/exim.conf. You can find out the actual name by running the following command:
exim -bP configure_file
On a system where Exim is fully installed as a replacement for Sendmail, one or both of the paths /usr/lib/sendmail or /usr/sbin/sendmail is a symbolic link to the Exim binary. Therefore, any MUA, program, or script that attempts to send a message by calling Sendmail actually calls Exim.
BSD-based systems tend to use /usr/sbin/sendmail, whereas Solaris uses /usr/lib/sendmail. Different MUAs have different defaults, so some administrators set both paths to cater for both kinds.
Whenever Exim is executed, it starts by reading its runtime configuration file. A large number of settings can be present, but for any one installation only a few are normally used. The data from the file is held in main memory while an Exim process is running. For this reason, if you change the file, you have to tell the Exim daemon to reload it. This is done by sending the daemon a SIGHUP signal. All other Exim processes are short-lived, so as new ones start up after the change, they pick up the new configuration.
For very simple installations, it may be possible to include all the configuration data within the runtime configuration file. A minimal usable configuration of this type is shown in the next chapter, in "A Minimal Usable Configuration File". Normally, however, the runtime configuration refers to auxiliary data, which can be in ordinary files, or in databases such as NIS or LDAP. Common examples are the system alias file (usually called /etc/aliases) and users' .forward files. Files or databases can also be used for lists of hosts, domains, or addresses that are to be handled in some special way and that are too long to conveniently include within the configuration file itself. Data from such sources is read afresh every time it is needed, so updates take immediate effect and there is no need to send a SIGHUP signal to the daemon.
The simplest item that is found in the runtime configuration file is an option set to a fixed string. For example, the following line:
qualify_domain = example.com
specifies that addresses containing only a local part and no domain are to be turned into complete addresses (''qualified'') by appending @example.com. Each such setting appears on a line by itself. For many option settings, fixed data suffices, but Exim also provides ways for you to supply data that is re-evaluated and modified every time it is used. Examples and explanations of this feature are introduced later in this chapter.
Unqualified addresses are accepted only from local processes, or from certain designated remote hosts.
How Exim Delivers Messages
Exim's configuration determines how it processes addresses; this processing involves finding information about the destinations of a message and how to transport it to those destinations. In this and the following sections, we discuss how the configuration that you set up controls what happens.
There are many different ways an address can be processed. For example, looking up a domain in the DNS involves a completely different way of processing from looking up a local part in an alias file, and delivering a message using SMTP over TCP/IP has very little in common with appending it to a mailbox file. There are separate blocks of code in Exim for doing the different kinds of processing, and each is separately and independently configurable. The word driver is used as the general term for one of these code blocks. In many cases, when you specify that a particular driver is to be used, you need only give one or two parameters for it. However, most drivers have a number of other options whose defaults can be changed to vary their behavior.
There are four different kinds of drivers. Three of them are concerned with handling addresses and delivering messages, and are called directors, routers, and transports. The fourth kind of driver handles SMTP authentication and is described in Chapter 15, "Authentication, Encryption, and Other SMTP Processing".
Transports are the components of Exim that actually deliver messages by writing them to files, or to pipes, or over SMTP connections. Directors and routers are very similar in that their job is to process addresses and decide what deliveries are to take place. The difference between them is in the kinds of address that they handle; directors handle local addresses and routers handle remote addresses. As Exim has evolved, the original differences in concept between directors and routers have diminished, and it may come about that they are merged in some future release. For the moment, however, a distinction remains.
Before going into more detail, we take a brief look at the way drivers are used as a message makes its way through the system. Exim has to decide whether each address is to be delivered on the local host or to a remote one, then it has to choose the right form of transport for each address (appending to a user's mailbox, for instance, or connecting to another host via SMTP), and finally it has to invoke those transports. For example, in a typical configuration, a message addressed to firstname.lastname@example.org, where exim.example is a local domain, might be handled like this:
The first driver in the configuration is a director that handles system aliases; this tells Exim to check the /etc/aliases file. Here it finds that the local part bug_reports is indeed an alias, and that it resolves to two other addresses: the local address email@example.com, and the remote address firstname.lastname@example.org. Further drivers must be invoked to handle each of these new recipients.
Later in the configuration is a director that recognizes local users like brutus, and it arranges for Exim to run a transport called appendfile, which adds a copy of the message to Brutus' mailbox. The actual delivery does not take place until after Exim has worked out how to handle all the addresses.
For the other recipient, Exim runs a router that looks up the domain helpersys.org.example in the DNS, and finds the IP address of the remote host to which the message should be sent. It then arranges for Exim to run the smtp transport in order to do the delivery.
This example has introduced several of the most commonly used drivers. Later in this chapter, we work through a similar example in much more detail. The individual drivers are described in their own sections in later chapters; here is an alphabetical list of them:
A director that expands aliases into one or more different addresses.
A transport that writes messages to local files.
A transport that generates automatic replies to messages.
A router that routes remote domains using locally supplied information.
A director that handles users' .forward files and Exim filter files.
A router that handles ''IP literal'' addresses such as email@example.com. These are relics of the early Internet that are no longer in common use.
A transport that delivers messages to external processes using the LMTP protocol.
LMTP (RFC 2033) is a variation of SMTP that is designed for passing messages between local processes.
A director that recognizes local usernames.
A router that looks up remote domains in the DNS.
A transport that passes messages to external processes via pipes.
A router that runs an external program in order to route a domain.
A director that accepts any address; it is used as a ''catchall.''
A transport that writes messages to other hosts over TCP/IP connections, using either SMTP or LMTP.
The configuration may refer to the same driver code more than once, but with different options, in order to create multiple instances of the same driver type. Each driver instance is given an identifying name in the configuration file, for use in logging and for reference from other drivers.
Local and Remote Addresses
There are two distinct types of mail address: those for which the local part is used when deciding how to deliver the message, and those for which only the domain is relevant. Typically, when a domain refers to a remote host, the local part of the address plays no part in the routing process, but if the domain is the name of the local host, the local part is usually used in determining where to deliver the message. This is not a hard and fast rule (a small company might accept mail for any local part in a single mailbox), but it forms the basis of the distinction between directors and routers.
The first thing Exim does when processing an address is to determine whether it should be handled by the directors or by the routers. An Exim configuration normally contains definitions of a number of directors and at least one router, though there may be any number of either. If the domain is listed in the configuration as a local domain, the address is processed by the directors and is called a local address. Otherwise it is processed by the routers and is called a remote address.
Exim decides whether a domain is local by checking the local_domains option, which contains a colon-separated list of patterns. If it is not set, the name of the local host is used as the only local domain. Otherwise, it may contain various types of patterns, of which the most common are shown in this example:
local_domains = tiber.rivers.example:\ *.cities.example:\ dbm;/usr/exim/domains
The first item in the list is a single domain name, tiber.rivers.example, while the second is a simple pattern, matching all domains that end in .cities.example. The third item is a reference to an external file, /usr/exim/domains, which is a DBM-keyed file. This type of item is useful when a host is handling a very large number of local domains. We discuss DBM files and this kind of lookup item in more detail later.
More complicated patterns can be given in the form of regular expressions.
Notice the use of backslashes for continuing the option value over several lines. This is a general feature of Exim's configuration file; any line can be continued in this way. Whitespace at the start of continuation lines is ignored.
In versions of Exim prior to 3.14, this continuation mechanism is available only in macro definitions, rewriting rules, and option settings where the value is given enclosed in double quotes. Thus, the earlier example would have to be quoted if used in an earlier version.
Processing an Address
After it has decided whether an address is local or remote, Exim offers it to each configured director or router (as appropriate) in turn, in the order in which they are defined, until one of them is able to deal with it. The order in which directors and routers are defined in the configuration file is therefore important. The process of directing a local address is illustrated in Figure 3-2; a similar process happens using the routers for a remote address.
Figure 3-2. Directing a local address
A director that successfully handles an address may add that address to a queue for a particular transport. Alternatively, it may generate one or more ''child'' addresses that are added to the message's address list and processed in their own right, with the original address no longer playing any part. This is what happens when a local part matches an entry in an alias list, or when a user's .forward file is activated.
A successful router, on the other hand, can only add the address to a queue for a transport, or modify the domain and pass it on to the next router. It cannot generate ''child'' addresses. When a director or a router cannot handle an address, it is said to decline. If every director or router declines, the address cannot be handled at all, and delivery fails.
Figure 3-3. Routing and directing
The way addresses are handled by directors and routers is illustrated in Figure 3-3. (The line labeled ''local after all'' is a special case that is discussed in "Remote Address Becoming Local", later in this chapter.) All the addresses in a message, and any that are generated from them (for example, by aliasing), are processed by the directors and routers before any deliveries take place from the transport queues. Any router or director can queue an address for any transport; directors are not restricted to local transports, nor routers to remote ones.
A Simple Example
To help clarify the mechanisms described earlier, an example of a simple message delivery is presented here. The scenario is a host called simple.example, where the hostname is the only local mail domain. The host is using a simple Exim configuration file that supports aliases, user-forward files, delivery to local users' mailboxes, and remote SMTP delivery. The relevant portions of the configuration are quoted here. Suppose a user of this host has sent a message addressed to one local and one remote recipient:
At the start of delivery, Exim's list of addresses to process is initialized with the two original recipients, and its first job is to work through this list, deciding what to do for each address. For firstname.lastname@example.org, the domain is local, so it is passed to the first defined director, whose configuration is as follows:
system_aliases: driver = aliasfile file = /etc/aliases search_type = lsearch
The first line, terminated by a colon, is the name for this particular director instance, chosen by the system administrator. Each driver of a particular type (director, router, or transport) must have a distinct name. However, names of driver instances can be the same as the names of the drivers themselves; you can have the following:
aliasfile: driver = aliasfile file = /etc/aliases search_type = lsearch
if you want to, but some people find this usage confusing. The second configuration line specifies which kind of director this is (or, to put it another way, it chooses which block of director code to run), and the remaining two lines are options for the director.
The aliasfile director handles an address by looking up the local part in an alias list, and the options control how the lookup is done. In this case, the list is in the file /etc/aliases, and a linear search (''lsearch'') is required. This expects each line of the file to contain an alias name, optionally terminated by a colon, followed by the list of replacement addresses for the alias, which may be continued onto subsequent lines by starting them with whitespace. A comma is used to separate addresses in the list. For example:
root: email@example.com, firstname.lastname@example.org postmaster: email@example.com
Notice that the first line specifies that root is an alias for postmaster, which itself is an alias. This is a common practice, and works exactly as you might expect. The aliasfile director reads through this file and finds the entry for postmaster, so it adds a new address, firstname.lastname@example.org, to the list of addresses to process, and returns a code that indicates success, meaning that email@example.com has been completely processed. The list of pending addresses now contains the following:
See "Directing Loops", later in this chapter, for a discussion of how it might go wrong.
Exim proceeds to tackle firstname.lastname@example.org,which is another local address, so again it is offered to the system_aliases director. This time, however, there is no match in /etc/aliases, so the director cannot handle the address. It returns a code indicating ''decline,'' which causes Exim to offer the address to the next director, whose configuration is as follows:
In practice, it might not actually happen in this order.
userforward: driver = forwardfile file = .forward
The job of a forwardfile director is to check for the existence of files containing lists of forwarding addresses. This instance is configured to look for .forward files in users' home directories. First of all, it has to check that the local part of the address corresponds to a user login name. If there is no matching user, the director declines, but if simon is in fact a user of the host, the director goes on to check the existence of the given file.
It does this by calling the system function getpwnam() rather than looking at /etc/passwd directly, so that users defined by other means (such as NIS) are recognized.
If the file is defined using a relative pathname, as shown earlier, it is sought in the user's home directory. Because home directories are often NFS-mounted, Exim first checks that the directory is available before trying to open the file so that the absence of the directory is not mistakenly interpreted as the absence of the file.
In an automounted environment, the directory check causes an automount to occur.
If simon has a .forward file, its contents are a list of forwarding addresses and other types of items, as described in "Items in Alias and Forward Lists", in Chapter 7, "The Directors". The addresses are added to the list of addresses to process, the userforward director returns a code indicating success, and the new addresses are eventually processed independently.
If simon does not have a .forward file, the director declines, and email@example.com is offered to the third director in the configuration:
localuser: driver = localuser transport = local_delivery
The job of localuser is to check whether the local part of the address corresponds to a user login name. In this configuration, this check has already been done by the previous director. This is quite a common occurrence, so Exim keeps a cache of the most recently looked-up name to avoid wasteful repetition. If simon were not a local user, the director would decline, and as there are no more directors in the configuration, the address would fail. It would be placed on a list of failed addresses and used to generate a bounce message at the end of the delivery attempt.
When the local user does exist, the director succeeds, and it places the address on a queue for the local_delivery transport, attaching to it the uid, gid, and home directory that it looked up. That is all that happens at this stage; no actual delivery takes place until later. The processing of firstname.lastname@example.org is illustrated in Figure 3-4, where the ovals represent sources of information, and the rectangles represent drivers.
Figure 3-4. Directing example
There is still one address to process: email@example.com. Its domain is not a local one, so it is processed by routers rather than by directors. Exim offers it to the first router:
lookuphost: driver = lookuphost transport = remote_smtp
This is in fact the only router in this simple configuration, so if it declines, the address fails. The job of lookuphost is to obtain a list of remote hosts for the domain of an address, and in its normal configuration (as shown earlier), it does this by looking up the domain in the DNS using MX and address records, as described in "DNS Records Used for Mail Routing", in Chapter 2, "How Internet Mail Works". When it is successful, it ends up with an ordered list of hosts and their IP addresses. It puts the mail address on a queue for the remote_smtp transport, attaching the host list. In our example, if the MX and address records were the following:
another.example. MX 6 mail-2.another.example. another.example. MX 4 mail-1.another.example. mail-1.another.example. A 192.168.34.67 mail-2.another.example. A 192.168.88.32
then the list of hosts to be passed with the address to remote_smtp would be:
mail-1.another.example 192.168.34.67 mail-2.another.example 192.168.88.32
Any hosts that have the same MX preference value are sorted into a random order. The processing of firstname.lastname@example.org is illustrated in Figure 3-5.
Figure 3-5. Routing example
There are now no more unprocessed addresses, so the directing and routing phase of the delivery process is complete, and Exim moves on to do the actual deliveries by running the transports that have been set up. Local transports are run first; in our example, there is one local delivery setup for the address email@example.com, using the local_delivery transport. This was specified by a localuser director that handled the address. The transport is configured thus:
local_delivery; driver = appendfile file = /var/mail/$local_part delivery_date_add envelope_to_add return_path_add
This uses the appendfile driver, which adds a copy of the message to the end of a mailbox file in conventional Unix format when configured in this way.
Other configurations (see "The appendfile Transport", in Chapter 9, "The Transports") support different formats.
The name of the file is given by the file option. Its value, with an embedded dollar character, is different from the option settings that we have met so far, which have all been fixed values. Much of the flexibility of Exim's configuration comes from the use of option settings where the specified strings are changed each time they are used. This process is called string expansion, and we'll see it in many examples throughout this book. A complete description of all the expansion features is given in Chapter 17, "String Expansion".
The simplest change that can be made to a string is the insertion of a variable value, and this is what is happening in the earlier example. Exim replaces teh substring $local_part by the local part of the address that is being delivered, so the file that is actually used is /var/mail/simon. The remaining three options request the addition of three generally useful header lines as the message is written:
A header that records the date and time of delivery, for example:
Delivery-Date: Fri, 31 Dec 1999 23:59:59 +0000
A header that records the original recipient address (the ''envelope to'' address) that caused the delivery; in this example it would be:
Preserving this address is useful in case it does not appear in the To: or Cc: headers.
A header that records the sender from the message's envelope, for example:
For bounce messages that have no sender, it looks like this:
Local deliveries are always run sequentially in separate processes that change their user identity to some specific value. In this case, the user ID (uid) and group ID (gid) of the local user were passed to the transport by the localuser director, so these are used. The delivery subprocess is therefore running ''as the user'' when it accesses the mailbox.
The use of different uids and gids in Exim is discussed in "Security Issues", in Chapter 19, "Miscellany".
When the subprocess has finished, there are no more local deliveries, so Exim proceeds to the remote ones. Before it does so, it gives up its root privilege permanently, and runs as the Exim user if a uid and gid for Exim have been defined in the configuration (either at build time or at runtime). This is the recommended way to run Exim.
There is one remote delivery, for firstname.lastname@example.org, which was set up by the lookuphost router to use the remote_smtp transport:
remote_smtp: driver = smtp
There are no option settings here beyond the one that selects the type of transport, because the list of hosts was obtained by the lookuphost router and passed to the transport along with the address. The parameters of the outgoing SMTP call (for example, the timeouts) can be changed by other options, but in this case we accept all the defaults. The smtp transport tries to make an SMTP connection to each host in turn. If all goes well, a connection is made to one of them, and the message is transferred.
There are now no more deliveries to be done, and all the recipients have been successfully handled, so at this point Exim can delete the message files on its spool and log the fact that this message has been delivered. The delivery process then exits.
The fragments of configuration file used in this example have been shown in the order in which they are used during delivery. The actual configuration file defines the transports first, followed by the directors, and finally the routers. The transports come first, so that when Exim is reading the file, they are defined before the director and router configurations that refer to them. In the following chapter, we show a complete configuration file.
Complications While Directing and Routing
Things do not always go as smoothly as described in the simple example. These are some of the more common complications that can be encountered when directing or routing an address.
Duplicate addresses are a complication that Exim may have to handle, either because the sender of the message specified the same address more than once, or because aliasing or forwarding duplicated an existing recipient address. For any given address, only a single delivery takes place, except when the duplicates are pipe commands. If one user is forwarding to another, and a message is sent to both of them, only a single copy is delivered. If, on the other hand, two different users set up their .forward files to pipe to /usr/bin/vacation (for example), a message that is sent to both of them runs the vacation program twice, once as each user.
Sometimes, a director or router is unable to determine whether it can handle an address. For example, if the administrator has misspelled the name of an alias file, or if it has been accidentally deleted, an aliasfile director cannot operate. Timeouts can occur when a router queries the DNS, and both routers and directors can refer to databases that may at times be offline. In these situations, the director or router returns a code indicating ''defer'' to the main part of Exim, and the address is neither delivered nor bounced, but left on the spool for another delivery attempt at a later time. The control of retry times is described in Chapter 12, "Delivery Errors and Retrying". If the error condition is felt to be sufficiently serious, the message is ''frozen,'' which means that queue runner processes will not try to deliver it. As frozen messages are highlighted in queue listings, this also serves to bring it to the administrator's attention.
When an aliasfile or forwardfile director handles an address, the new addresses that it generates are each processed afresh, just like the original recipient addresses. This means that one alias can refer to another, as in the example we showed earlier:
This is the normal practice; there are occasions when it is not wanted, and there is an option, new_director, that can be used to disable it.
root: email@example.com postmaster: firstname.lastname@example.org
However, it opens up the possibility of directing loops. To prevent this, Exim automatically skips a director if the address it is handling has a ''parent'' address that was processed by that director. Consider the following broken alias file:
chicken: email@example.com egg: firstname.lastname@example.org
This director turns a message addressed to email@example.com into firstname.lastname@example.org, and then turns it back into email@example.com the next time through. However, on the third pass, Exim notices that the address was previously processed by the director, so it is skipped and the next director is called. The chances are that the resulting delivery or bounce are not what was intended, but at least the loop is broken.
Remote Address Becoming Local
It sometimes turns out that when a router is processing an address, it discovers that the domain is a local domain after all. This can happen if the domain was originally given in an abbreviated form (for example, as in the address brutus@rome), because DNS lookups are commonly configured to expand single- component names into the full form, within the local encompassing domain. If routing changes the domain name, and the result is a local domain, the address is automatically passed from the router to the directors.
Remote Address Routing to the Local Host
After Exim has routed a remote address, it checks to see whether the first host on the list of hosts to which the message could be sent is the local host. Usually, this indicates some kind of configuration error, and by default Exim treats it as such. However, there are types of configuration where it is legitimate, and for these cases the self option can be used to pass such addresses from the router to the directors.
See, for example, "Mixed Local/Remote Domains", in Chapter 5, "Extending the Delivery Configuration".
Complications During Delivery
A successful routing process for a remote address discovers a list of hosts to which it can be sent, but it cannot check the local part of the address. The most common permanent error during a remote delivery is ''unknown user,'' which is given in response to an SMTP RCPT command. Responsibility for the message remains with the sending host, which must return a bounce message to the sender.
Not all receiving hosts behave like this; some accept any local part (in their local domain) during the SMTP dialog, and do the check later. By this time, responsibility for the message has been passed, so it is the receiving host that has to generate the bounce. When Exim is a receiving host, it can be configured to act in either manner, depending on the setting of receiver_verify and related options (see "Verifying Recipient Addresses", in Chapter 13, "Message Reception and Policy Controls").
There are other reasons a remote host might permanently refuse a message, and in addition, there are many common temporary errors, such as the inability to contact a host. These cause a message to remain on the spool for later delivery.
In contrast to routing, directors for local addresses normally check local parts, so any ''unknown user'' errors happen at directing time. The only problems a local transport is likely to encounter are errors in the actual copying of the message. The most common is a full mailbox; Exim respects system quotas and can be configured to impose its own quotas (see "Mailbox Quotas", in Chapter 9, "The Transports"). A quota failure leaves the message on the spool for later delivery.
The runtime configuration contains a set of retry rules (see Chapter 12, "Delivery Errors and Retrying") that specify how often, and for how long, Exim is to go on trying to deliver messages that are suffering temporary failures. The rules can specify different behaviors for different kinds of error.
Complications After Delivery
When all delivery attempts for a message are complete, a delivery process has two final tasks. If any deliveries suffered temporary errors, or if any deliveries succeeded after previous temporary errors, the delivery process has to update the retry hints database. This work is saved up for the end of delivery so that the process opens the hints database for updating only once at most, and for as short a time as possible. If the updating should fail, the new hint information is lost, but previous hint information remains. In practice, except in exceptional circumstances such as a power loss, hint information is rarely lost.
Finally, unsuccessful delivery may cause a message to be sent to the sender. If any addresses failed, a single bounce message is generated that contains information about all of them. If any addresses were deferred, and have been delayed for more than a certain time (see "Delay Warning Messages", in Chapter 19, "Miscellany"), a warning message may be sent.
Exim sends such messages by calling itself in a subprocess. Failure to create a bounce message causes Exim to write to its panic log and immediately exit. This has the effect of leaving the message on the spool so that there will be another delivery attempt, and presumably another attempt at sending the bounce message when the delivery fails again. Failure to create a warning message, on the other hand, is not treated as serious. Another attempt to send it is made when the original message is processed again.
Use of Transports by Directors and Routers
In the simple example we have been considering, the localuser director and the lookuphost router include the transport option, referring to the local_delivery and remote_smtp transports, respectively, whereas the other directors do not have any transport settings. A transport is required for any router or director that actually sets up a message delivery to determine how the delivery should be done. When a director is just changing the delivery address by aliasing or forwarding, a transport is not required because no delivery is being set up at that stage.
Depending on their configurations, some directors and routers require a transport setting, and some require there is not a transport setting. Exim detects an incorrect configuration when the configuration file is read. In other cases, the director or router may behave differently, depending on whether or not a transport is supplied. These variations are explained in the detailed descriptions of the directors and routers (see Chapter 7, "The Directors" and Chapter 8, "The Routers").
Two directors, aliasfile and forwardfile, have additional options for special-purpose transports. These directors can deliver a message to a specific file, or to a pipe associated with a given command. For example, a line in an alias file of the form:
majordomo: |/usr/mail/majordomo ...
specifies that a message addressed to the local part majordomo is to be passed via a pipe to a process running the command:
The other entries in the alias file may just be changing delivery addresses, and therefore may not require a transport. However, this line is setting up a delivery, and so a transport is required. We can add to the system_aliasesdirector configuration the following line, which in our example runs the aliasfile director:
address_pipe_transport = alias_pipe
This tells Exim which transport to run when a pipe is specified in the alias file. The transport itself is very simple:
alias_pipe: transport = pipe ignore_status return_output
A pipe transport runs a given command in a new process, and passes the message to it using a pipe for its standard input. In this example, the command is provided by the alias file, so the transport does not need to define it. Setting ignore_status tells Exim to ignore the status returned by the command; without this, any value other than zero is treated as an error, causing the delivery to fail and a bounce message to be returned to the sender.
If the pipe transport is run directly from a director or router, the command to be run is defined using its command option.
Setting return_output changes what happens if the command produces output on its standard output or standard error streams. By default, such output is discarded, but if return_output is set, the production of such output is treated as an error, and the output itself is returned to the sender in the bounce message.
There is one piece of information that the pipe transport needs that we have not yet given, and that is the uid and gid under which it should run the command. When a pipe is triggered by an entry in a user's .forwardfile, the user's identity is assumed by default, but when an alias file is used, as it is here, there is no default. The user (and, optionally, group) option can appear in either the director or the transport's configuration, so the transport could become:
This assumes that all the pipes specified in the alias file are to be run under the same uid. If there are several instances that require different user identities, an expansion string can be used to select the correct uid, but that is too advanced for the discussion here.
alias_pipe: transport = pipe ignore_status return_output user = majordom
In addition to delivery to pipes, alias files and forward files may also specify specific files into which messages are to be delivered. For example, if user caesar has a .forward containing:
it requests delivery to another mail address, and also into the named file, which is a delivery that needs a transport. To support this feature, the userforward director could contain:
address_file_transport = address_file
This tells Exim which transport to run when a filename is specified instead of an address in a forward file. The transport itself is even more simple than the pipe transport:
address_file: driver = appendfile
The filename comes from the forward file, and all other options are defaulted.
An alias or forward file may contain both of these kinds of entries, thus requiring both address_pipe_transport and address_file_transport to be given on a single director. These options are used for these very specific purposes only, and should not be confused with the generic transport option that applies to all directors and routers.
Back to: Exim: The Mail Transfer Agent
© 2001, O'Reilly & Associates, Inc.