ftp.pasteur.org/FAQ/

home *** CD-ROM | disk | FTP | other *** search

/ ftp.pasteur.org/FAQ/ / ftp-pasteur-org-FAQ.zip / FAQ / mail / filtering-faq < prev next >

Wrap

Internet Message Format | 1996-10-17 | 48.8 KB

Path: senator-bedfellow.mit.edu!bloom-beacon.mit.edu!eru.mt.luth.se!news.kth.se!newsfeed.sunet.se!news01.sunet.se!sunic!02-newsfeed.univie.ac.at!newsfeed.uk.ibm.net!arclight.uoregon.edu!feed1.news.erols.com!howland.erols.net!cs.utexas.edu!swrinde!nntp.primenet.com!news1.best.com!nntp1.best.com!shellx!ii From: FAQ Editor <faq-editor@ii.com> Newsgroups: comp.mail.misc,comp.mail.elm,comp.mail.mh,comp.mail.mush,comp.mail.pine,comp.mail.zmail,comp.answers,news.answers Subject: Filtering Mail FAQ Followup-To: comp.mail.misc Date: 4 Oct 1996 23:37:54 -0400 Organization: Infinite Ink Lines: 1364 Sender: ii@best.com Approved: news-answers-request@mit.edu Message-ID: <ii.844486542@shellx> Reply-To: FAQ Editor <faq-editor@ii.com> NNTP-Posting-Host: shellx.best.com X-Newsreader: NN version 6.5.0 #6 (NOV) Xref: senator-bedfellow.mit.edu comp.mail.misc:35371 comp.mail.elm:23304 comp.mail.mh:9308 comp.mail.mush:2574 comp.mail.pine:20995 comp.mail.zmail:1410 comp.answers:21568 news.answers:83562 Archive-name: mail/filtering-faq Posting-frequency: approximately monthly Last-modified: 4 October 1996 Copyright: See section 7.0 FAQ-launcher-URLs: http://www.jazzie.com/ii/faqs/archive/mail/filtering-faq/ http://www.best.com/~ii/faqs/archive/mail/filtering-faq/ All current hypertext versions of this FAQ are accessible through either of the above two "FAQ Launcher" URLs. If you want to link to this FAQ, please link to these launchers rather than to only one version of the FAQ (e.g., rather than linking to the version you are viewing right now!). This will give your users a choice of which version they access, e.g., a single-file version or an index-plus-multiple-files version. Also, please give links to both mirrors of the launcher since sometimes one or the other is down. _____________________________________________________ ________| |________ \ | FILTERING MAIL FAQ | / \ | Copyright (c) 1994-96 Nancy McGough and others | / / |_____________________________________________________| \ /___________) (__________\ TABLE OF CONTENTS 0.0 Preliminaries 0.1 Getting the Latest Version of this FAQ 0.1.1 Hypertext 0.1.2 Plain Text 0.2 Terminology 0.3 Notation 1.0 Mail Folder Strategies 1.1 Naming Incoming Mail Folders 1.2 Reading Incoming Mail Folders 2.0 Procmail 2.1 Setting Up Procmail 2.1.1 Is Procmail on Your System? 2.1.2 Setting Up Procmail for Testing 2.1.3 Testing 2.1.4 Setting Up Procmail to Filter Mailing List Messages 2.2 Troubleshooting Procmail 2.2.1 General Strategies 2.2.2 Alternate .forward files 2.3 Explanation of Test Recipe 2.4 Tracking Your Incoming Mail 2.5 Procmail References 3.0 Mailagent 3.1 Setting Up Mailagent 3.2 Tracking Your Incoming Mail 3.3 How Safe Mailagent Processing Is? 3.4 Locking Under Mailagent 3.5 Folder Types Supported 3.6 Mailagent References 4.0 Filter 4.1 Setting up Filter 4.2 Tracking Your Incoming Mail 4.3 Filter References 5.0 Mailer and Newsreader References 6.0 Contributors 6.1 Acknowledgements 6.2 Contributing to this FAQ 7.0 Copyright Notice ---------------------------------------------------------------------- From: FAQ Editor <faq-editor@ii.com> Subject: 0.0 Preliminaries Q: How can I have my incoming mailing-list messages automatically put into appropriate folders? This is one of the most frequently asked questions about email. This FAQ provides basic instructions for Unix users to set up either procmail, mailagent, or elm's filter to filter incoming mailing-list messages. If you have a choice, I recommend procmail which is powerful, robust, and is actively developed and supported. All these mail processors can also be set up to filter other types of incoming messages, send automated replies, etc. For instructions see the relevant references listed below. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 0.1 Getting the Latest Version of this FAQ If this FAQ is over a couple months old, there may be an updated version. Please get the latest hypertext or plain text version from one of the places listed below. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 0.1.1 Hypertext The best way to read this FAQ (and most other FAQs too) is to view the hypertext version using a Web browser such as Cello, Lynx, Mosaic, Netscape Navigator, Microsoft Internet Explorer, OmniWeb, SpiderWoman or WinWeb. This will allow you to easily jump: * between subjects in the FAQ * to any Uniform Resource Locator (URL) in the FAQ * to an Internet Request For Comments document (RFC) * to some manual pages This, and all FAQs that are crossposted to news.answers, are available at the following hypertext FAQ archives: http://www.cs.ruu.nl/cgi-bin/faqwais/ http://www.lib.ox.ac.uk/search/search_faqs.html http://www.smartpages.com/bngfaqs/top.html All hypertext versions of this particular FAQ are accessible through either of these "FAQ Launcher" URLs: http://www.jazzie.com/ii/faqs/archive/mail/filtering-faq/ http://www.best.com/~ii/faqs/archive/mail/filtering-faq/ ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 0.1.2 Plain Text The plain text version is regularly posted to comp.mail.misc, comp.mail.elm, comp.mail.mh, comp.mail.mush, comp.mail.pine, comp.mail.zmail, comp.answers, and news.answers. It's in digest format which means that you may be able to use your newsreader to easily move between digest items (e.g., nn uses G% to burst a digest and trn uses ^G to jump to the next digest item). The plain text FAQ is also available through: Anonymous FTP: ftp://rtfm.mit.edu/pub/usenet/news.answers/mail/filtering-faq Email: Send mail to: mail-server@rtfm.mit.edu Containing the following in the body of the message: send usenet/news.answers/mail/filtering-faq UUCP: uunet!/archive/usenet/news.answers/mail/filtering-faq Hard Copy: A printed version of this FAQ is in Chapter 25 of the book "Internet Secrets" by John R. Levine and Carol Baroudi; published 1995 by IDG Books; ISBN 1-56884-452-2. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 0.2 Terminology Term Meaning ==== ======= mailer or MUA Mail user agent such as pine or elm pico PIne COmposer - a user friendly editor reader Mailer, newsreader, or Web browser that can read mail folders regular expression Text that can include "wild cards" (such as . to match any single character); used for searching ^x Press the Ctrl key and then, while holding down the Ctrl key, press the x key ~ or $HOME Your home directory. You can always get to your home directory by typing `cd'. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 0.3 Notation Notation Meaning ======== ======= TextName placeholder; replace with appropriate text <text> placeholder; replace with appropriate text without the angle brackets `text' text you type but without the quotes "text" text you type including the double quotes 'text' text you type including the single quotes [Key] press the key, e.g., [Space], [Enter], or [Tab] ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 1.0 Mail Folder Strategies It is a good idea to come up with a system for naming and reading your mail folders. With a good system, you will have an easier time managing the hundreds (or thousands!) of messages you will receive. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 1.1 Naming Incoming Mail Folders For my incoming mail folders I use names that start with `IN'. For example, I put mail sent to the procmail mailing list into a folder named IN.procmail. This way when all my folders are listed alphabetically the incoming folders are together and near the top. They are near the top because Unix is case sensitive and upper case letters come before lower case letters in an ascii sort. Of course, you can use any names you like for your mail folders! ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 1.2 Reading Incoming Mail Folders Here are some instructions for opening a mail folder that's in the default folder directory for some mailers and newsreaders. Default Folder Reader Directory Command line command From within reader ====== ============== ==================== ================== elm ~/Mail elm -f =folder c =folder pine ~/mail pine -if folder G folder mail ~/mail mail -f /path/folder nn ~/News nn +folder G +folder (Please send me information for other mailers and newsreaders.) Pine has an incoming-folder variable which you can use to list your incoming folders in a separate section of your folder list (to view your folder list in Pine, type `L'). Setting this variable also allows you to use the Tab key to step through all new messages in all your incoming folders. For more information see the question "How can I filter messages into different incoming folders?" in the Pine FAQ: http://www.cac.washington.edu/pine/QandA/FAQs.html#xtocid178164 IMPORTANT NOTE ============== If you are going to be editing your incoming mail folder, e.g., deleting messages, then your filtering program and your folder reader should use locking schemes that work together. Otherwise, if mail is delivered at the same time that you are deleting a message your folder may be corrupted. To learn more about locking see the question "What is folder locking and how does it work?" in the Pine FAQ. This is useful to read even if you don't use Pine as your mailer. http://www.cac.washington.edu/pine/QandA/sysadmins.html#xtocid7145 Most mailers, such as pine and elm, use lock a file named `folder.lock' to lock the folder named `folder'. Note that nn does not lock folders so you probably don't want to use nn to delete messages in a folder that is receiving incoming messages. (Please send me info about locking in other mailers and newsreaders.) For more information about specific mailers and newsreaders, see the relevant documentation, such as man pages and Web pages. Some pointers to documentation are listed in section 5.0. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.0 Procmail Procmail is a powerful mail processor that can be used to process your mail messages either as they arrive or after they are in a mail folder. To find out how to process an existing mail folder see the NOTES section of the procmail(1) man page. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.1 Setting Up Procmail To set up and test procmail, follow the ten steps given in sections 2.1.1 through 2.1.4 below. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.1.1 Is Procmail on Your System? 1] To find out if procmail is on your system and what the full path to it is, type one of the following commands: Shell Command ===== ======= csh which procmail sh or ksh type procmail various whereis procmail various where procmail Make a note of the full path to procmail because this is needed in step 5b below. You can find out the version by typing: procmail -v The latest released version, as of 15 April 1995, is 3.10. Version 3.11 should be out soon. If your system doesn't have procmail or doesn't have the latest version, you may want to ask your system administrator to install it. The procmail package of tools is at: ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail/procmail.tar.gz If your sys admin isn't able to do this, use a different mail processor like mailagent (described in section 3 of this FAQ) or filter (described in section 4). ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.1.2 Setting Up Procmail for Testing 2a] Create ~/.procmailrc. cd pico .procmailrc NOTE: Throughout this article I use pico for editing files. Replace `pico' with your editor. 2b] Enter a modified version of the following in your ~/.procmailrc. Note that lines that begin with # are comments and are ignored by procmail. #Set on when debugging VERBOSE=off #Replace `mail' with your mail directory (Pine uses mail, Elm uses Mail) MAILDIR=$HOME/mail #Directory for storing procmail log and rc files PMDIR=$HOME/.procmail LOGFILE=$PMDIR/log INCLUDERC=$PMDIR/rc.testing INCLUDERC=$PMDIR/rc.maillists 3] Create the directory where you will store your procmail log and rc files (this is $PMDIR that you set above). cd mkdir .procmail 4a] Create an rc (run commands) file for testing: cd .procmail pico rc.testing 4b] Enter the following in ~/.procmail/rc.testing: :0: * ^Subject:.*test IN.testing Note that the first line contains a zero (0), not the letter "oh". For now, don't worry about the meaning of this recipe. It is explained in 2.3 "Explanation of Test Recipe." 5a] Create a ~/.forward file by typing the following. (Pico's -w flag tells pico not to auto wrap lines.) cd pico -w .forward 5b] Enter a modified version of the following in your ~/.forward: "|IFS=' ' && exec /usr/local/bin/procmail -f- || exit 75 #nancym" == IMPORTANT NOTES == * Make sure you include all the quotes, both double (") and single ('). * The vertical bar (|) is a pipe. * Replace /usr/local/bin with the correct path for procmail (see step 1). * Replace `nancym' with your userid. You need to put your userid in your .forward so that it will be different than any other .forward file on your system. * Do NOT use ~ or environment variables, like $HOME, in your .forward file. If procmail resides below your home directory write out the *full* path. 5c] On many systems you need to make your .forward world readable and your home directory world searchable in order for the mail transport agent to "see" it. To do this type: cd chmod 644 .forward chmod a+x . ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.1.3 Testing 6] Send yourself two test messages: one with `test' in the subject and one without `test' in the subject. 7] Start your mailer (pine, elm, etc.) and check that the messages were delivered correctly. The one with `test' in the subject should be in the folder $MAILDIR/IN.testing and the one without `test' in the subject should be in your inbox. If these were not delivered correctly, see section 2.2 on "Troubleshooting Procmail" section below. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.1.4 Setting Up Procmail to Filter Mailing List Messages 8a] Once you have successfully tested procmail in steps 6 and 7, create rc.maillists for filtering incoming mailing-list messages into mail folders. cd cd .procmail pico rc.maillists 8b] In rc.maillists, create a recipe, like the two example recipes below, for each of your mailing lists. :0: * ^TOwww-talk IN.www-talk :0: * ^TOprocmail IN.procmail The first recipe filters the www-talk mailing list and the second recipe filters the procmail mailing lists. The meaning of the first recipe is as follows: Notation Meaning ======== ======= :0 Begin a recipe : Use a lock file * Begin a condition ^TO Match ``To:'' ``Cc:'' or other synonyms for To at the beginning of a line, followed by any or no characters, followed by.... www-talk ``www-talk'' IN.www-talk If successful match, put in folder $MAILDIR/IN.www-talk IMPORTANT NOTES =============== * ^TO is not a normal regular expression; it is a special procmail expression that is designed to catch any destination specification. For details, see the MISCELLANEOUS section of the procmailrc(5) man pages. * Do not put a space between the caret (^) and the word `TO' in `^TO'. * Do not put a space between the `^TO' and the text that you are matching on; it must be `^TOtext'. * Both letters in `TO' must be capitalized. 9] Repeat steps 6 and 7 to make sure that things are still working. 10] Comment out the rc.testing line in you .procmailrc file so that it looks like this: VERBOSE=off MAILDIR=$HOME/mail PMDIR=$HOME/.procmail LOGFILE=$PMDIR/log # INCLUDERC=$PMDIR/rc.testing INCLUDERC=$PMDIR/rc.maillists NOTE: Leaving the rc.testing line in your .procmailrc file is useful for future testing. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.2 Troubleshooting Procmail If messages are not delivered correctly, here are steps you can use to try to solve the problem. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.2.1 General Strategies 1] Look at your $LOGFILE (~/.procmail/log) to see if you can determine what the problem is. 2] Check these three files for typos: ~/.forward ~/.procmailrc ~/.procmail/rc.testing 3] Check the file and directory permissions of your .forward (set in 5c in "Setting up Procmail for Testing" above). Type... In order to... ------- -------------- cd Go to your home directory. ls -l .forward Check the permission: it should say -rw-r--r-- ls -ld . Check permission of home dir: it should say drwx?-x?-x The ?'s may be r's or hyphens or one of each (i.e., drwx--x--x, drwxr-xr-x, drwxr-x--x, drwx--xr-x are each acceptable.) 4] If the above three steps do not locate the problem edit your ~/.procmailrc so that it contains: VERBOSE=on Test procmail by following steps 6 and 7 again. Look at your $LOGFILE (which will contain verbose messages) to see if you can now determine what the problem is. If you are still having problems see the next section on "Alternate .forward Files." After you get procmail to work, you probably will want to set VERBOSE back to off. ------------------------------ From: "Stephen R. van den Berg" <berg@pool.informatik.rwth-aachen.de> Subject: 2.2.2 Alternate .forward files If the .forward template in 5b above doesn't work the following alternatives might be helpful: In a perfect world: "|exec /usr/local/bin/procmail #nancym" In an almost perfect world: "|exec /usr/local/bin/procmail USER=nancym" In another world: "|IFS=' ';exec /usr/local/bin/procmail #nancym" In a different world: "|IFS=' ';exec /usr/local/bin/procmail USER=nancym" In a smrsh world: "|/usr/local/bin/procmail #nancym" These formats can be tried in different combinations, the leading "| can be tried as |" instead, or vice versa. NOTE ==== If your system uses procmail as its local delivery agent, you do not need a .forward file; simply having a .procmailrc files suffices. To find out if procmail is the local delivery agent, ask your system administrator. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.3 Explanation of Test Recipe The recipe you used for testing is: :0: * ^Subject:.*test IN.testing The meaning of this recipe is: Notation Meaning ======== ======= :0 Begin a recipe : Use a lock file * Begin a condition ^ Match the beginning of a line followed by.... Subject: ``Subject:'' followed by.... . any character (.) followed by.... * 0 or more of preceding character (any character in this case) followed by.... test ``test'' IN.testing If successful match, put in folder $MAILDIR/IN.testing ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.4 Tracking Your Incoming Mail You can use mailstat, a useful script that is part of the procmail package, to check your procmail log file. Check to see if it is on your system by typing either `which mailstat' or `type mailstat'. If it's on your system type: mailstat $HOME/.procmail/log This displays a concise version of your log file and moves your log file to log.old. You may want to put the above line in your .login so that each time you log in you will see a listing of how many messages you've received since the last time you ran mailstat, and what folders these messages were delivered to. You can get a mailstat listing of log.old by using the -o flag: mailstat -o $HOME/.procmail/log If mailstat is not on your system ask your system administrator to install it. The script is located with all the other procmail tools (see 2.1.1 above for the ftp location). ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 2.5 Procmail References Web Page: http://www.jazzie.com/ii/internet/procmail/ http://www.best.com/~ii/internet/procmail/ (mirror of above URL) Man Pages: procmail(1) - autonomous mail processor procmailrc(5) - procmail rc file procmailex(5) - procmail rc file examples procmailsc(5) - procmail weighted scoring techique egrep(1) - search file for regular expression (procmail uses egrep-style regular exprssions along with some of its own expressions like ^TO) formail(1) - mail reformatter sendmail(8) - send mail over the internet Newsgroup: comp.mail.misc Mailing List: procmail@informatik.rwth-aachen.de Subscribe to the procmail mailing list by sending mail: To: procmail-request@informatik.rwth-aachen.de Subject: subscribe Procmail Archives: Get a list of files available at the procmail mail server by sending mail: To: procmail-request@informatik.rwth-aachen.de Subject: archive ls Get Best of the Procmail mailing list by sending mail (you'll need gzip and a MIME-decoder to unpack it): To: procmail-request@informatik.rwth-aachen.de Subject: archive get best_of_procmail_list* ------------------------------ From: Raphael Manfredi <ram@hptnos02.grenoble.hp.com> Subject: 3.0 Mailagent Mailagent is a powerful mail processing package that can be used to process your mail messages, either at arrival time via a .forward hook or later on while they are already saved in a mail folder. Mailagent is written in Perl and hence has all the advantages of being interpreted, i.e. it is easy to enhance dynamically and to customize. This version of the FAQ describes the basics of setting up mailagent to process incoming mail messages from within a .forward. To find out about the more advanced features of mailagent, please refer to the mailagent(1) manual page. ------------------------------ From: Raphael Manfredi <ram@hptnos02.grenoble.hp.com> Subject: 3.1 Setting Up Mailagent 1] First, make sure mailagent is available on your system. The easiest way to do this is to run: mailagent -V which will print the mailagent version if it is available, or the shell will issue an error message "mailagent: not found" or something like it. 2a] Now you need to understand the MTA (Mail Transport Agent, the program that delivers the mail; usually, sendmail) will NOT deliver to mailagent directly, rather to an intermediate (small) filter program. Two versions are available: shell or C -- refer to the mailagent(1) manpage to choose, but I recommend you use the C version first, and move to the shell version if you can't run a binary from your .forward file. 2b] Locate the filter program (it will be filter or filter.sh depending on whether you choose the C or the shell version respectively) under some directory like /usr/local/lib/mailagent. From now on, we'll assume we use the C filter and that it is located under /usr/local/lib/mailagent. 3] Copy the file /usr/local/lib/mailagent/mailagent.cf as ~/.mailagent and edit it to configure your system correctly. You will see two distinct sections in that file and you need to set-up properly the first one, the "Configuration section". If you have a version of mailagent that is recent enough (at least 3.0 PL32) then you can create an initial configuration very easily and quickly by running the following command: mailagent -I which will set-up an almost ready-to-use ~/.mailagent file. All you need to do at this point is go through its configuration section to make sure mailagent made the right choices... The minimal set of variables that MUST be correctly set (i.e. for which you cannot rely on the default set in the file) are (# introduces comment in a shell-like manner, which run up to the end of the line): home: # Your HOME directory, as reported by "echo $HOME", usually. path: # Path to be used to locate mailagent and perl, at least. p_xxx: # Path to be appended to "path" above when running on machine "xxx". user: # Your login name. name: # Your name. level: # Logging level. I recommend you raise it to 20 for testing. The mail folder directory is ~/Mail by default, but it can be changed only from within your rule file by putting maildir = ~/mail; at its top, for instance, to make it ~/mail. The rule file is defined as the "rules" parameter, and is set to ~/.rules by default. 4] Ensure the directories configured in your ~/.mailagent under "logdir" and "spool" and "queue" do exist. If you use the standard setting, this requires the following commands: cd mkdir var cd var mkdir log mailagent cd mailagent mkdir queue 5a] Create a rule file (named ~/.rules by default) for testing: cd vi .rules 5b] Enter the following in ~/.rules Subject: /test/ { SAVE testing }; The meaning of that rule should be pretty obvious: If we receive a mail whose subject line contains the word "test", then we save that mail in a folder named "testing", under the default folder directory (~/Mail). 5c] Create a ~/.forward file as follows: "|exec /usr/local/lib/mailagent/filter >> /export/home/ram/.bak 2>&1" The meaning of that line is the following: every mail should be piped (hence the leading "|" character) onto the filter program, and any output from that program (i.e. errors) are appended to some file in your home directory, with stderr following stdout (2>&1) in traditional sh syntax. == IMPORTANT NOTES == * Your .forward is always processed by sh, regardless of your login shell. * Replace /export/home/ram with your proper login directory full path. That's a part that makes your .forward unique (for zealous optimizing sendmail that are dead wrong about optimizing!) and that can save you a lot of trouble if anything goes wrong! Just look at your ~/.bak! * Replace /usr/local/lib/mailagent/filter with the proper filter path on your machine. 5d] Note that on many systems, you need to ensure your .forward will be readable by everybody, and that your home directory has the "x" bit set for everybody (i.e. can be part of a lookup path) so that sendmail can see and parse your .forward file. To ensure this, type: cd chmod a+r .forward chmod a+x . 6] Send yourself two test messages: one with "test" in the subject, and one without "test" in the subject. 7a] Checkout your ~/.bak file: it should be empty! 7b] Checkout your ~/var/log/agentlog file to see what really happened to your messages. Watch out for any ERROR or WARNING logs. This assumes you have left at least the default logging level (9) in the ~/.mailagent file (the "level" variable). But for testing, that level should be raised to 20 to help you diagnose what's going on. 7c] Look out in ~/Mail/testing. You should find there the message whose Subject line contained the word "test". Then make sure the other message has been delivered to your regular mailbox. (Since no match occurred in your rule file, the mail is left in your mailbox by default). 7d] TROUBLESHOOTING * If your mail was not properly delivered, please make sure your rule file and configuration file are correct by issuing the following command: mailagent -d * If the previous command does not output the single rule you should have put in ~/.rules, then please make sure step 3 and 4 above were correctly performed (those being the crucial steps ensuring a proper configuration). * Check the ~/.bak file for error messages. * Check for typos in any of these files: ~/.forward ~/.mailagent ~/.rules * Check the file and directory permissions of your .forward (set in step 5c above). Type... In order to... ------- -------------- cd Go to your home directory. ls -l .forward Check the permission: it should say -rw-r--r-- ls -ld . Check permission of home dir: it should say drwx?-x?-x The ?'s may be r's or hyphens or one of each (i.e., drwx--x--x, drwxr-xr-x, drwxr-x--x, drwx--xr-x). * If none of the previous hints helped you identify problem, and you can't figure it out from the output in ~/.bak or in the ~/var/log/agentlog file (or whatever file you have configured for logging within your ~/.mailagent file, variables "log" and "logdir"), then make sure your mail is not waiting in the MTA's queue: this might be the case if the agentlog file is empty. If you are using sendmail as MTA, you can run: /usr/lib/sendmail -bp to print out the queue. * As a last resort, please look at the mailagent(1) manual page under the section "Testing Your Installation" for more tips and things to look at. 8] Once you have successfully tested mailagent in steps 6 and 7 above, you're on! Mailagent is ready to process your mail. All you have to do is extend the ~/.rules file to add more rules. For instance: To Cc: www-talk { SAVE www-talk }; To Cc: agent-users { SAVE agent-users }; Those two rules filter the two mailing lists www-talk and agent-users into their respective folders, whether the mailing list address appear in the To OR Cc header. Since rules are not qualified as a pattern match (contrary to our test above), they match on logins in the address, i.e. they will match things like www-talk@chip.com or chip!www-talk, or a plain simple www-talk if this is a local alias. (This implicit matching on logins works only for some selectors like To, Cc or From which are know to contain addresses). If you wish to sort on patterns appearing in the Subject of messages for instance, then you must use a pattern matching syntax, as in: Subject: /star trek/ { SAVE star-trek; }; to save in a folder "star-trek" all the messages whose subject contains the words "star trek". Case matters, but keep on reading... 9] As an advanced topic, since mailagent is written in Perl, you have all the power of Perl's regular expressions at your disposal. Which means you can write things like this: To Cc: agent-users { REJECT AGENT }; <AGENT> Subject: /^\w*subscribe/i { DELETE }; <AGENT> * { UNIQUE -a; SAVE agent }; The second lines makes use of that perl extended regular expression syntax: \w matches an alphanumeric character plus "_", while the trailing "i" option requests a case-insensitive match. You also note you have a real automaton at your disposal. You can enter a special state (AGENT in our example) and continue parsing by only scanning for rules tagged by this mode. The first match stops the automaton, unless you REJECT to continue processing. When not restricted by a mode list, a rule is always taken into account. For example, assuming the automaton is in the state "NEWS", it will not consider rules tagged <AGENT>, as in the above example. The automaton begins in mode "INITIAL". The "UNIQUE -a" action followed by a SAVE ensures only one copy per Message-ID will ever end-up in your agent folder -- no duplicates! Also note you can have more than one action per rule, and that the last one uses '*' to match anything, i.e. its action part between {} will always be executed in AGENT mode, when reached by the automaton. Also, since in Perl regular expression syntax, \b matches a word-boundary and \s any space or tab character, we can write our Star Trek message sorting into a much more robust form: Subject: /\bstar\s+trek\b/i { SAVE star-trek; }; (\s+ matches one or more white spaces, while \s* would match zero or more, see the Perl manual page for a complete description of regular expressions.) which will match on various subject strings like "Last Star Trek season" or "I am addicted to Star trek", but not on "Tristar treks" -- whatever that means :-) All in all, the filtering automaton syntax is pretty much intuitive and easy to read. You have to learn which actions are possible and what they mean, naturally. ------------------------------ From: Raphael Manfredi <ram@hptnos02.grenoble.hp.com> Subject: 3.2 Tracking Your Incoming Mail If you are curious about what mailagent does to your mail, you have two options: * Look at your "agentlog" file, with a log level set to 9. * Get a summary of all the actions performed by running: mailagent -summary This last option is only possible if you have initialized the statistics gathering process by creating a ~/var/mailagent/mailagent.st file (under the default setting from ~/.mailagent). You will get a clear picture of your processing, by seeing which rule match, how often, in which state, etc... You will also know how many times you SAVE or DELETE messages for instance. Actually, mailagent statistics are triggered by a simple mailagent -s command, the letters given after in the {u,m,a,r,y} set being options that alter the output given by the command. * Look at the ~/.bak occasionally to make sure no error is pending... ------------------------------ From: Raphael Manfredi <ram@hptnos02.grenoble.hp.com> Subject: 3.3 How Safe Mailagent Processing Is? As soon as the filter program has taken a hold on your message, you can rest assured the mail will get filtered one way or the other. If filter can't queue your mail, it will exit with an exit status of 75, that status being recognized by "sendmail" as a "deliver later on" hint, in which case the mail message will safely wait in sendmail's queue. So if filter gets your message, it immediately forks and exits with a 0 status for sendmail, letting it know its work is finished and releasing it to save resources. It then calls mailagent on the queued message (in mailagent's private queue) to actually process the message. Only after successful processing will mailagent delete the queued message. Hence, under an heavily loaded system, the worst that could happen would be a duplicate processing of a message, or a bounce back when sendmail cannot fork and exec the filter program from your .forward. Under catastrophic conditions, filter or mailagent will simply dump the message on stdout, for ~/.bak to catch, preceded by the reason why processing was aborted. ------------------------------ From: Raphael Manfredi <ram@hptnos02.grenoble.hp.com> Subject: 3.4 Locking Under Mailagent By default, mailagent proceeds with a fixed locking scheme (.lock extension) plus flock() if asked to do so at Configure time. However, mailagent provides support for NFS-secure locks and also can use non-standard locking procedures, configurable from within ~/.mailagent (variables "nfslock" and "mboxlock"). However, it cannot support locking on a rule basis (yet!). The author is willing to raise the priority of that item if one comes up with a legitimate need for that feature that could not be worked-around by a PERL escape. ;-) ------------------------------ From: Raphael Manfredi <ram@hptnos02.grenoble.hp.com> Subject: 3.5 Folder Types Supported Mailagent can deliver mail to plain UNIX folders (also known as "mbox format"), to MMDF folders, to MH folders (with unseen sequence update built-in and using locks, not like rcvstore which does not!) or to directories (ala MH, but without unseen sequence support and with alternate naming possible). It also supports delivery to folders with the "x" bit set, in which case mailagent interprets those as being hooks. It either pipes the message to the "program" or further interprets the hook to do more processing. See the manual page mailagent(1) under the section "MAIL HOOKS" for more details. ------------------------------ From: Raphael Manfredi <ram@hptnos02.grenoble.hp.com> Subject: 3.6 Mailagent References Man Pages mailagent(1) - reference manual page (about 47 pages troff'ed). perl(1) - reference manual for Perl sendmail(8) - send mail over the Internet Examples: agent/examples/rules - a commented rule file sample, from the distribution source tree. Newsgroup: comp.mail.misc Mailing List: agent-users@foretune.co.jp Subscribe to the list by sending mail: To: majordomo@foretune.co.jp subscribe agent-users Mailagent archives: ftp://ftp.foretune.co.jp/pub/network/mail/mailagent EMAIL: Send mail to the author's mailagent: To: ram@hptnos02.grenoble.hp.com Subject: Command @SH maildist - mailagent - ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 4.0 Filter Filter is part of the Elm package of tools. Note that you can use filter to filter your incoming mail even if you are not using Elm to read your mail. IMPORTANT NOTE ============== If your system has both procmail and filter installed then you should use procmail which is *much* more robust and powerful than filter. This recommendation is almost universal; even the developers of Elm and Filter recommend procmail over filter. IT IS POSSIBLE TO LOSE MAIL MESSAGES WHEN USING FILTER; this is rare but it has happened. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 4.1 Setting up Filter Followup-To: comp.mail.elm 1] Find out if filter is on your system and what the full path to it is. If you are using the C shell (csh) type: which filter Or, if you are using the Korn shell (ksh) or the Bourne shell (sh) type: type filter If neither ``which'' nor ``type'' are on your system try ``where'' and ``whereis''. If your system doesn't have filter ask your system administrator to install it; or even better ask her to install procmail. 2] Note the full path of your home directory by typing: cd pwd 3a] Create ~/.elm/filter-rules. (Note that throughout this article I use pico for editing files. Replace ``pico'' with your editor.) cd mkdir .elm cd .elm pico filter-rules 3b] Enter a modified version of the following in your ~/.elm/filter-rules if (subject contains "test") then save "/j/nancym/Mail/IN.testing" Replace /j/nancym with your home directory path (see step 2). Replace /Mail with the name of the directory where your mail folders are stored. Pine and Berkeley Mail use /mail (lower case m) and Elm uses /Mail (upper case M). 4] To see what the filter rule will do type the following at your Unix prompt: filter -r 5a] Create a ~/.forward file by typing the following. (Pico's -w flag tells pico not to auto wrap lines.) cd pico -w .forward 5b] Enter a modified version of the following in your ~/.forward: "|/usr/local/bin/filter -o /j/nancym/.elm/filter-errors" == IMPORTANT NOTES == * Make sure you include the quotes ("). * The vertical bar (|) is a pipe. * Replace /usr/local/bin with the correct path for filter (see step 1). * Replace /j/nancym with your home directory (see step 2). * Do NOT expect environment variables, like $HOME, to work in your .forward file. * Do NOT expect ~ to mean your home directory in the .forward file. 5c] Note that on many systems you need to make your .forward is world readable and your home directory world searchable in order for the mail transport agent to "see" it. To do this type: cd chmod 644 .forward chmod a+x . 6] Send yourself two test messages: one with ``test'' in the subject and one without ``test'' in the subject. 7a] Start your mailer (pine, elm, etc.) and check that the messages were delivered correctly. The one with ``test'' in the subject should be in the folder IN.testing and the one without ``test'' in the subject should be in your inbox. If these were delivered correctly go on to step 8. 7b] TROUBLESHOOTING * If the two messages were not delivered correctly look at your ~/.elm/filter-errors to see if you can determine what the problem is. * Check these two files for typos: ~/.forward ~/.elm/filter-rules * Check the file and directory permissions of your .forward (set in 5c above). Type... In order to... ------- -------------- cd Go to your home directory. ls -l .forward Check the permission: it should say -rw-r--r-- ls -ld . Check permission of home dir: it should say drwx?-x?-x The ?'s may be r's or hyphens or one of each (i.e., drwx--x--x, drwxr-xr-x, drwxr-x--x, drwx--xr-x). * If none of these turn up the problem edit your ~/.forward so that filter will be verbose with it's output (use the -vo flag). "|/usr/local/bin/filter -vo /j/nancym/.elm/filter-errors" And repeat steps 6 and 7. After you get filter to work you will probably want to change the ``-vo'' flag back to ``-o''. 8] After you have successfully tested filter in steps 6 and 7, edit ~/.elm/filter-rules so that it contains a modified version of the following: # if (subject contains "test") then save "/j/nancym/Mail/IN.testing" if (to contains "www-talk") then save "/j/nancym/Mail/IN.www-talk" if (to contains "hopos-l") then save "/j/nancym/Mail/IN.hopos" Replace /j/nancym with your home directory path and /Mail with the name of your mail directory. Replace the mailing list string (e.g., "www-talk") and the name of the mail folder (e.g., IN.www-talk) with text for your mailing lists. Note that ``to contains...'' means either the To or Cc header contains... Rather than deleting the test line, it's useful to just turn it into a comment (by preceding it with #) so that you can easily use it for future testing. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 4.2 Tracking Your Incoming Mail Followup-To: comp.mail.elm You can get a short summary of filter's activity by typing: filter -s For a longer summary type: filter -S Or you can look at the log file itself, ~/.elm/filterlog. You should regularly look at ~/.elm/filter-errors to make sure things are working. You can automatically check filter-errors each time you log in by putting the following in your .login: tail ~/.elm/filter-errors Also you probably want to regularly delete filterlog and filter-errors so they don't fill up your disk space. To get a summary of the filter log and clear it type: filter -cs ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 4.3 Filter References Followup-To: comp.mail.elm Web Pages: http://www.myxa.com/elm.html http://www.math.fu-berlin.de/~guckes/elm/elm.filter.html Man Pages: filter(1) Newsgroup: comp.mail.elm ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 5.0 Mailer and Newsreader References PINE ==== Web Pages: http://www.cac.washington.edu/pine/ http://www.jazzie.com/ii/internet/pine/ http://www.best.com/~ii/internet/pine/ (mirror of above URL) Man Pages: pine(1), pico(1), pilot(1) Newsgroup: comp.mail.pine (linked to Pine mailing list) Mailing List: pine-info@cac.washington.edu (linked to Pine newsgroup) Subscribe to the pine-info mailing list by sending mail to: majordomo@cac.washington.edu With... subscribe pine-info in the body of the message. ELM === Web Pages: http://www.myxa.com/elm.html http://www.math.fu-berlin.de/~guckes/elm/ Man Pages: elm(1) Newsgroup: comp.mail.elm Emacs Mail Mode =============== Newsgroups: gnu.emacs.help and comp.emacs MH == Man Pages: mh(1) Newsgroup: comp.mail.mh MAIL ==== Man Pages: mail(1) Newsgroup: comp.mail.misc NN == Web Pages: http://www.jazzie.com/ii/internet/nn/ http://www.best.com/~ii/internet/nn/ (mirror of above URL) Man Pages: nn(1) Newsgroup: news.software.nn Please send me pointers to other mailer and newsreader references and let me know what newsreaders can read mail folders. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 6.0 Contributors This FAQ, like many others, is a collaborative effort. I learned a lot of the information in newsgroups and mailing lists, especially: comp.mail.* procmail mailing list Also, lots of people have mailed me information and I've tried to acknowledge them below. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 6.1 Acknowledgements Thanks to these people who sent suggestions: David L. Miller <dlm@cac.washington.edu> Cookie Monster <kmanley@bits.fc.hp.com> Jim Showalter <gamma@mintaka.disa.mil> David W. Tamkin <dattier@mcs.com> Rick Troxel <rick@helix.nih.gov> Stephen R. van den Berg <berg@pool.informatik.rwth-aachen.de> Syd Weinstein <syd@dsinc.myxa.com> Special thanks to: Thomas A. Fine <fine@cis.ohio-state.edu> for setting up and maintaining the hypertext archive of FAQs. Congratulations to him for winning O'Reilly and Associates' "The Best of the Net" award! Please let me know if I've left you, or anyone else, out of this list. ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 6.2 Contributing to this FAQ If you have any corrections, suggestions, or new digest items to contribute to this FAQ please send them to faq-editor@ii.com. If your reader understands the following URL, you can use it to send me mail: mailto:faq-editor@ii.com. Things I'm especially interested in are: * a comparison of mail filtering tools * what newsreaders can read mail folders * instructions for opening a specific folder in various readers * lock scheme and files used by various readers * other Unix filtering tools, like deliver * filtering tools for other operating systems like the Mac and DOS/Windows ------------------------------ From: FAQ Editor <faq-editor@ii.com> Subject: 7.0 Copyright Notice This FAQ is Copyright (c) 1994-96 Nancy McGough, except sections 2.2.2 and 3.0 through 3.6 which are Copyright (c) 1994-96 by the authors named in those sections. No portion of this work may be sold or put to commercial use without express written consent of the authors. This restriction covers publication in any form, or distribution by any method, which permits this work to be visually perceived, either directly or with the aid of any machine or device. Permission is granted to republish or redistribute this article in its entirety for noncommercial use if this copyright notice is not removed or altered. For more information, see Infinite Ink's Legal Notice, which is available at: http://www.jazzie.com/ii/legalese/ http://www.best.com/~ii/legalese/ (mirror of above URL) End of Filtering Mail FAQ Digest ******************************** -- <*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*<*< @..@ Nancy McGough /\_/\ (----) Infinite Ink ( o.o ) ( >__< ) http://www.jazzie.com/ii/ > ~ <