-- $Id: FAQ,v 3.0.1.4 1995/08/07 16:04:41 ram Exp $ -- -- Copyright (c) 1990-1993, Raphael Manfredi -- -- You may redistribute only under the terms of the Artistic License, -- as specified in the README file that comes with the distribution. -- You may reuse parts of this distribution only within the terms of -- that same Artistic License; a copy of which may be found at the root -- of the source tree for mailagent 3.0. -- -- $Log: FAQ,v $ -- Revision 3.0.1.4 1995/08/07 16:04:41 ram -- patch37: updated my e-mail address -- -- Revision 3.0.1.3 1995/02/16 14:19:48 ram -- patch32: now mentions new mailagent option -I for quick setup -- patch32: made some cleanup following Nancy McGough's comments -- -- Revision 3.0.1.2 1995/02/03 17:54:03 ram -- patch30: detab-ed the FAQ file for easier reading on terminals -- patch30: minor changes in wording and added the Star Trek example -- -- Revision 3.0.1.1 1995/01/03 17:53:38 ram -- patch24: created -- [This FAQ written for inclusion in the "Filtering Mail FAQ" posting] Subject: 5.0 Mailagent Mailagent is a powerful mail processing package that can be used to process your mail messages, either at arrival time via a .forward hook or later on while they are already saved in a mail folder. Mailagent is written in Perl and hence has all the advantages of being interpreted, i.e. it is easy to enhance dynamically and to customize. This version of the FAQ describes the basics of setting up mailagent to process incoming mail messages from within a .forward. To find out about the more advanced features of mailagent, please refer to the mailagent(1) manual page. ----------------------------------- Subject: ... 5.1 Setting Up Mailagent 1] First, make sure mailagent is available on your system. The easiest way to do this is to run: mailagent -V which will print the mailagent version if it is available, or the shell will issue an error message "mailagent: not found" or something like it. 2a] Now you need to understand the MTA (Mail Transport Agent, the program that delivers the mail; usually, sendmail) will NOT deliver to mailagent directly, rather to an intermediate (small) filter program. Two versions are available: shell or C -- refer to the mailagent(1) manpage to choose, but I recommend you use the C version first, and move to the shell version if you can't run a binary from your .forward file. 2b] Locate the filter program (it will be filter or filter.sh depending on whether you choose the C or the shell version respectively) under some directory like /usr/local/lib/mailagent. From now on, we'll assume we use the C filter and that it is located under /usr/local/lib/mailagent. 3] Copy the file /usr/local/lib/mailagent/mailagent.cf as ~/.mailagent and edit it to configure your system correctly. You will see two distinct sections in that file and you need to set-up properly the first one, the "Configuration section". If you have a version of mailagent that is recent enough (at least 3.0 PL32) then you can create an initial configuration very easily and quickly by running the following command: mailagent -I which will set-up an almost ready-to-use ~/.mailagent file. All you need to do at this point is go through its configuration section to make sure mailagent made the right choices... The minimal set of variables that MUST be correctly set (i.e. for which you cannot rely on the default set in the file) are (# introduces comment in a shell-like manner, which run up to the end of the line): home: # Your HOME directory, as reported by "echo $HOME", usually. path: # Path to be used to locate mailagent and perl, at least. p_xxx: # Path to be appended to "path" above when running on machine "xxx". user: # Your login name. name: # Your name. level: # Logging level. I recommend you raise it to 20 for testing. The mail folder directory is ~/Mail by default, but it can be changed only from within your rule file by putting maildir = ~/mail; at its top, for instance, to make it ~/mail. The rule file is defined as the "rules" parameter, and is set to ~/.rules by default. 4] Ensure the directories configured in your ~/.mailagent under "logdir" and "spool" and "queue" do exist. If you use the standard setting, this requires the following commands: cd mkdir var cd var mkdir log mailagent cd mailagent mkdir queue 5a] Create a rule file (named ~/.rules by default) for testing: cd vi .rules 5b] Enter the following in ~/.rules Subject: /test/ { SAVE testing }; The meaning of that rule should be pretty obvious: If we receive a mail whose subject line contains the word "test", then we save that mail in a folder named "testing", under the default folder directory (~/Mail). 5c] Create a ~/.forward file as follows: "|exec /usr/local/lib/mailagent/filter >> /export/home/ram/.bak 2>&1" The meaning of that line is the following: every mail should be piped (hence the leading "|" character) onto the filter program, and any output from that program (i.e. errors) are appended to some file in your home directory, with stderr following stdout (2>&1) in traditional sh syntax. == IMPORTANT NOTES == * Your .forward is always processed by sh, regardless of your login shell. * Replace /export/home/ram with your proper login directory full path. That's a part that makes your .forward unique (for zealous optimizing sendmail that are dead wrong about optimizing!) and that can save you a lot of trouble if anything goes wrong! Just look at your ~/.bak! * Replace /usr/local/lib/mailagent/filter with the proper filter path on your machine. 5d] Note that on many systems, you need to ensure your .forward will be readable by everybody, and that your home directory has the "x" bit set for everybody (i.e. can be part of a lookup path) so that sendmail can see and parse your .forward file. To ensure this, type: cd chmod a+r .forward chmod a+x . 6] Send yourself two test messages: one with "test" in the subject, and one without "test" in the subject. 7a] Checkout your ~/.bak file: it should be empty! 7b] Checkout your ~/var/log/agentlog file to see what really happened to your messages. Watch out for any ERROR or WARNING logs. This assumes you have left at least the default logging level (9) in the ~/.mailagent file (the "level" variable). But for testing, that level should be raised to 20 to help you diagnose what's going on. 7c] Look out in ~/Mail/testing. You should find there the message whose Subject line contained the word "test". Then make sure the other message has been delivered to your regular mailbox. (Since no match occurred in your rule file, the mail is left in your mailbox by default). 7d] TROUBLESHOOTING * If your mail was not properly delivered, please make sure your rule file and configuration file are correct by issuing the following command: mailagent -d * If the previous command does not output the single rule you should have put in ~/.rules, then please make sure step 3 and 4 above were correctly performed (those being the crucial steps ensuring a proper configuration). * Check the ~/.bak file for error messages. * Check for typos in any of these files: ~/.forward ~/.mailagent ~/.rules * Check the file and directory permissions of your .forward (set in step 5c above). Type... In order to... ------- -------------- cd Go to your home directory. ls -l .forward Check the permission: it should say -rw-r--r-- ls -ld . Check permission of home dir: it should say drwx?-x?-x The ?'s may be r's or hyphens or one of each (i.e., drwx--x--x, drwxr-xr-x, drwxr-x--x, drwx--xr-x). * If none of the previous hints helped you identify problem, and you can't figure it out from the output in ~/.bak or in the ~/var/log/agentlog file (or whatever file you have configured for logging within your ~/.mailagent file, variables "log" and "logdir"), then make sure your mail is not waiting in the MTA's queue: this might be the case if the agentlog file is empty. If you are using sendmail as MTA, you can run: /usr/lib/sendmail -bp to print out the queue. * As a last resort, please look at the mailagent(1) manual page under the section "Testing Your Installation" for more tips and things to look at. 8] Once you have successfully tested mailagent in steps 6 and 7 above, you're on! Mailagent is ready to process your mail. All you have to do is extend the ~/.rules file to add more rules. For instance: To Cc: www-talk { SAVE www-talk }; To Cc: agent-users { SAVE agent-users }; Those two rules filter the two mailing lists www-talk and agent-users into their respective folders, whether the mailing list address appear in the To OR Cc header. Since rules are not qualified as a pattern match (contrary to our test above), they match on logins in the address, i.e. they will match things like www-talk@chip.com or chip!www-talk, or a plain simple www-talk if this is a local alias. (This implicit matching on logins works only for some selectors like To, Cc or From which are know to contain addresses). If you wish to sort on patterns appearing in the Subject of messages for instance, then you must use a pattern matching syntax, as in: Subject: /star trek/ { SAVE star-trek; }; to save in a folder "star-trek" all the messages whose subject contains the words "star trek". Case matters, but keep on reading... 9] As an advanced topic, since mailagent is written in Perl, you have all the power of Perl's regular expressions at your disposal. Which means you can write things like this: To Cc: agent-users { REJECT AGENT }; Subject: /^\w*subscribe/i { DELETE }; * { UNIQUE -a; SAVE agent }; The second lines makes use of that perl extended regular expression syntax: \w matches an alphanumeric character plus "_", while the trailing "i" option requests a case-insensitive match. You also note you have a real automaton at your disposal. You can enter a special state (AGENT in our example) and continue parsing by only scanning for rules tagged by this mode. The first match stops the automaton, unless you REJECT to continue processing. When not restricted by a mode list, a rule is always taken into account. For example, assuming the automaton is in the state "NEWS", it will not consider rules tagged , as in the above example. The automaton begins in mode "INITIAL". The "UNIQUE -a" action followed by a SAVE ensures only one copy per Message-ID will ever end-up in your agent folder -- no duplicates! Also note you can have more than one action per rule, and that the last one uses '*' to match anything, i.e. its action part between {} will always be executed in AGENT mode, when reached by the automaton. Also, since in Perl regular expression syntax, \b matches a word-boundary and \s any space or tab character, we can write our Star Trek message sorting into a much more robust form: Subject: /\bstar\s+trek\b/i { SAVE star-trek; }; (\s+ matches one or more white spaces, while \s* would match zero or more, see the Perl manual page for a complete description of regular expressions.) which will match on various subject strings like "Last Star Trek season" or "I am addicted to Star trek", but not on "Tristar treks" -- whatever that means :-) All in all, the filtering automaton syntax is pretty much intuitive and easy to read. You have to learn which actions are possible and what they mean, naturally. ----------------------------------- Subject: ... 5.2 Tracking Your Incoming Mail If you are curious about what mailagent does to your mail, you have two options: * Look at your "agentlog" file, with a log level set to 9. * Get a summary of all the actions performed by running: mailagent -summary This last option is only possible if you have initialized the statistics gathering process by creating a ~/var/mailagent/mailagent.st file (under the default setting from ~/.mailagent). You will get a clear picture of your processing, by seeing which rule match, how often, in which state, etc... You will also know how many times you SAVE or DELETE messages for instance. Actually, mailagent statistics are triggered by a simple mailagent -s command, the letters given after in the {u,m,a,r,y} set being options that alter the output given by the command. * Look at the ~/.bak occasionally to make sure no error is pending... ----------------------------------- Subject: ... 5.3 How Safe Mailagent Processing Is? As soon as the filter program has taken a hold on your message, you can rest assured the mail will get filtered one way or the other. If filter can't queue your mail, it will exit with an exit status of 75, that status being recognized by "sendmail" as a "deliver later on" hint, in which case the mail message will safely wait in sendmail's queue. So if filter gets your message, it immediately forks and exits with a 0 status for sendmail, letting it know its work is finished and releasing it to save resources. It then calls mailagent on the queued message (in mailagent's private queue) to actually process the message. Only after successful processing will mailagent delete the queued message. Hence, under an heavily loaded system, the worst that could happen would be a duplicate processing of a message, or a bounce back when sendmail cannot fork and exec the filter program from your .forward. Under catastrophic conditions, filter or mailagent will simply dump the message on stdout, for ~/.bak to catch, preceded by the reason why processing was aborted. ----------------------------------- Subject: ... 5.4 Locking Under Mailagent By default, mailagent proceeds with a fixed locking scheme (.lock extension) plus flock() if asked to do so at Configure time. However, mailagent provides support for NFS-secure locks and also can use non-standard locking procedures, configurable from within ~/.mailagent (variables "nfslock" and "mboxlock"). However, it cannot support locking on a rule basis (yet!). The author is willing to raise the priority of that item if one comes up with a legitimate need for that feature that could not be worked-around by a PERL escape. ;-) ----------------------------------- Subject: ... 5.5 Folder Types Supported Mailagent can deliver mail to plain UNIX folders (also known as "mbox format"), to MMDF folders, to MH folders (with unseen sequence update built-in and using locks, not like rcvstore which does not!) or to directories (ala MH, but without unseen sequence support and with alternate naming possible). It also supports delivery to folders with the "x" bit set, in which case mailagent interprets those as being hooks. It either pipes the message to the "program" or further interprets the hook to do more processing. See the manual page mailagent(1) under the section "MAIL HOOKS" for more details. ----------------------------------- Subject: ... 5.6 Mailagent References Manuals: mailagent(1) - reference manual page (about 47 pages troff'ed). perl(1) - reference manual for Perl sendmail(8) - send mail over the Internet Examples: agent/examples/rules - a commented rule file sample, from the distribution source tree. Newsgroup: comp.mail.misc Mailing List: agent-users@foretune.co.jp Subscribe to the list by sending mail: To: majordomo@foretune.co.jp subscribe agent-users Mailagent archives: FTP://ftp.foretune.co.jp/pub/network/mail/mailagent EMAIL: Send mail to the author's mailagent: To: ram@hptnos02.grenoble.hp.com Subject: Command @SH maildist - mailagent - -----------------------------------