ftp.pasteur.org/FAQ//ftp-pasteur-org-FAQ.zip/FAQ/mail/procmail/pointers

Internet Message Format | 2004-05-05 | 25.9 KB

Path: senator-bedfellow.mit.edu!dreaderd!not-for-mail
Message-ID: <mail/procmail/[e-mail]>
Supersedes: <mail/procmail/[e-mail]>
Expires: 2 Jun 2004 12:58:04 GMT
X-Last-Updated: 1999/04/28
Organization: none
From: <[e-mail]> (Jari Aalto+mail.procmail)
Subject: Procmail pointers
Summary: This post contains excerpts from Procmail Tips page at
  which contains lot of infomation about Procmail, ube (aka spam),
  email headers, RFC headers etc. Athor's homepage is at
  http://poboxes.com/jari.aalto/
Newsgroups: comp.mail.misc,comp.answers,news.answers
Approved: [e-mail]
Followup-To: poster
Precedence: bulk
Originator: [e-mail]
Date: 04 May 2004 13:01:16 GMT
Lines: 535
NNTP-Posting-Host: penguin-lust.mit.edu
X-Trace: 1083675676 senator-bedfellow.mit.edu 567 18.181.0.29
Xref: senator-bedfellow.mit.edu comp.mail.misc:79466 comp.answers:57062 news.answers:270895

Archive-name: mail/procmail/pointers
Posting-Frequency: 2 times a month
Maintainer: Jari Aalto <[e-mail]>

Announcement: "Procmail resources"

        Last updated: 1999-04-21 v1.72 pm-tips.txt

        Faq archive is at

            http://www.faqs.org/faqs/

        The following list below is an excerpt from a bigger document.

            http://www.procmail.org/jari/pm-tips.html

        My Procmail module library page is at

            http://www.procmail.org/jari/pm-tips.html

        Era's exellent procmail link page (including procmail FAQ) is at

            http://www.iki.fi/~era/procmail/links.html

4.0 Procmail pointers

    4.1 Where to get procmail binary

        ftp://ftp.informatik.rwth-aachen.de/pub/packages/procmail/

    4.2 Where is procmail developed

        Philip Guenther <[e-mail]> is currently taking care of and
        coordinating procmail bug fixes. Please send any procmail bugs to
        the mailing list or to <[e-mail]>. The development mailing
        list is running SmarList at <[e-mail]>. Furher
        patch and bug info can be found at:

            http://www.gac.edu/~guenther/procmail/todo.html
            http://www.gac.edu/~guenther/procmail/warts.html

        Newest Procmail code:

            http://www.procmail.org/
            ftp://ftp.procmail.org/

    4.3 About procmail's Y2K compliance

        Please consult Philip Guenther <[e-mail]> for more up to date
        details. Philip is the Procmail maintainer currently.

        [1998-09-23 Bennett Todd <[e-mail]> in Message-Id:
        <[e-mail]>] Well, from a simple ogle of the
        grep over the sources, it looks like there may be a Y2038 problem
        in the autoconf test code: unsigned otimet = time(). And another,
        possibly less likely to express itself, in formail.c: unsigned long
        h1 = time(). Those could express themselves when 32-bit signed
        time_t wraps; long before then the time_t define should have been
        changed to something that is bigger, even it's "long long". The
        above type-mixes may fail to profit from a suitably redefined
        time_t, and so may overflow on 2038.

        I don't see any Y2K problems, though. And email headers use
        four-digit years pretty consistently, so that should all be cool.
        This estimation doesn't constitute an in-depth Y2k audit of
        procmail, but the source code to procmail is ... kinda dense for
        in-depth auditing.

        [1998-09-25 Bennett Todd Message-Id:
        <[e-mail]>] As I see it there are at least
        three measures that a whole email system, taken in aggregate, could
        use for Y2K checking. First, capture a vast cross-section of
        traffic and make sure no email software is using 2-digit years. I
        don't recall having seen any, but it's still worth checking.
        Second, generate a load of traffic with 2000 and 2001 dates and
        shove it through all the channels. And third, run all the systems
        end-to-end with their system clocks rolling over the millenium.

    4.4 Procmail mailing lists

        Traffic in this list is about 5-20 messages per day. Do not join
        if you can't handle that much traffic. The list is run by SmartList,
        which is a procmail based list software.

        ._MailingList_: questions/answers <[e-mail]>
        .subscription requests <[e-mail]>
        .digest request <[e-mail]>

       To get off the procmail mailing list

        To get off the list: send a message to *procmail-request* with:

            unsubscribe user@domain         in the subject line
            unsubscribe                     first line in the body

        If that fails, try email to
        <[e-mail]> (purportedly that should
        go to a person). See also the original subscriptions message that
        you will received http://www.iki.fi/~era/procmail/welcome.txt

    4.5 Procmail recipe modules and faqs

        Procmail is discussed in usenet newsgroup *comp.mail.misc*.

       "Procmail archive"
        ftp://ftp.informatik.rwth-aachen.de:/pub/packages/procmail/
        Articles from procmail mailing list: covers from 1994-08 to 1995-05
        (A .gz file: ~2Meg when uncompressed)

        And Latest articles can be found here, hosted by Achim Bohnet
        Covers from 1995-10 to the present day.
        <[e-mail]>. The www page has nice search capabilities.
        http://www.rosat.mpe-garching.mpg.de/mailing-lists/procmail/
        http://www.rosat.mpe-garching.mpg.de/~ach/exmh/archive/procmail/

       "Era's Procmail faq"
        http://www.iki.fi/~era/procmail/mini-faq.html
        http://www.dcs.ed.ac.uk/~procmail/faq/          [mirror]
        Also available by email, the ITEM can be: links.html, mini-faq.html,
        procmail-faq

            To: <[e-mail]>
            Subject: send ITEM

       "Era's Procmail Link collections"
        http://www.iki.fi/~era/procmail/links.html
        ...A page with full of good links to the world of procmail

       "Catherine's Getting Started With Procmail"
        http://shell3.ba.best.com/~ariel/nospam/proctut.shtml
        This is a quick tutorial intended to get a procmail neophyte
        started using procmail with as little trouble and fuss as possible.

       "Joe Gross's short Procmail tutorial"
        http://www.procmail.net/ <[e-mail]> ...Using procmail and a
        feature of ph you can set up your own mailing list without
        neededing root on your own machine.

       "Unix manpages"
        http://www.xs4all.nl/~pater/manpages/
        ...If you don't have procmail manpages at hand, check this site.
        It contains wealth of Unix related manpages online.
        <[e-mail]> Jeroen Paternostre

    4.6 Procmail Lint service (code check)

          _NOTE_: This service has closed 1999-04-20. It has been mentioned
          here for historical reasons only.

        If you have and can use Emacs, please download the Procmail
        programming mode, `tinypm.el', that [jari] has written. Lint is
        included in there and it can auto-correct mistakes on the fly. You
        can get it from the mentioned _uta_ ftp site (get tgz kit).

        Because not all people know how to use Emacs, how to use Emacs lisp
        packages or are otherwise clueless about Unix tools, I put up
        a procmail based Lint service where you can send your code.

            To: [e-mail]
            Subject: send pm-lint.hlp

        This service is highly experimental and if traffic starts to get
        too high, I have to close it because every message to the lint
        starts a background Emacs process and it consumes server
        resources. The preferred way is that you get your own Emacs package
        and Lint your code locally. When you send a message to the Lint
        it will respond to you with a message similar to this one:

            *** 1997-11-24 22:13 (pm.lint) 3.11pre7 tinypm.el 1.80
            cd /users/jaalto/junk/
            pm.lint:010: Warning, no right hand variable found. ([$`']
            pm.lint:055: Pedantic, flag orer style is not standard `hW:'
            pm.lint:060: Warning, message dropped to folder, you need lock.
            pm.lint:062: Warning, recipe with "|" may need `w' flag.
            pm.lint:073: Warning, Formail used but no `f' flag found.

    4.7 Procmail module list

       Where to get the modules

        The UBE stop procmail modules are not listed here. See pointers in
        "procmail code" section later.

        o   All pm-ja*.rc modules are in Jari's procmail kit. Also
            available ay Procmail code library page is at
            http://www.netforward.com/poboxes/?jari.aalto --> pm-code.html

        o   Other modules are by Alan Stbbens http://reality.sgi.com/aks/

        o   1998-12-08 Eli the Bearded <*@qz.to> announced in
            comp.mail.misc that he had put his own procmail modules to
            visible at http://www.qz.to/eli/src/procmail/. You may find
            interesting procmail code there but the modules itself are not
            general purpose *plug-in* modules that you could use right
            away. Some functionality included:

                Inline decoding of MIME text attachments        (rc.mime-decode)
                Cleansing of obscure "Re:" formats in subject   (rc.pre-list)
                Nifty autoresponder                             (rc.qz-2)
                Sophisticated dupicate email catching           (rc.dupes)
                Example of using my mail bouncer                (rc.lists-out)
                Detection of some classes of autoreplies        (rc.daemon)
                Various junk mail filtering                     (rc.filter)
                Daily log files                                 (rc.vars)

       Terminology

        *subroutine* = A piece of code that gets something in `INPUT' and
        responds with `OUTPUT'. Subroutine is not message specific.

        *recipe* = A piece of code, that is somewhat self containing:
        It reads something from the message or does something
        according to matches in message. Recipe may be message specific.

       Foreword to using modules

        In the module listing, some of the modules are recipes and some can
        be considered subroutines. Let's take the address explode module
        that was discussed a while ago. First, visualise following familiar
        programming language pseudo code:

        (ret-val1, ret-val2 ...) = Function( arg1, arg2, arg3 ...)

        *Function* may return multiple arguments and multiple arguments can
        be passed to it. Clear so far. Let's show how this applies to
        procmail modules:

            RC_FUNCTION  = $PMSRC/pm-xxx.rc # name the subroutine/module
            RC_FUNCTION2 = ...

            INPUT       = "value"           # Set the arg1 for module
            INCLUDERC   = $RC_FUNCTION      # Call Function( $arg1 )

            :0                              # Examine function ret val
            * ERROR ?? yes
            ...

        This should be pretty clear too. You just have to look into the
        subroutine/module, which you intend to use, to find out what
        arguments it wants which you _need_ _to_ set (INPUT) before calling
        it. The documentation also tells you what values are returned, eg.
        one of them was ERROR.

        If it were recipe/module, the call would be almost the same, but
        instead of returning values, the recipe/module most likely does
        something to your message or writes something to the data files or
        etc. *Recipe/module* is much higher level hierarchy, because it may
        call multiple subroutine/modules. The distinction between
        subroutine and recipe module type is not crystal clear, but I hope
        the above will clarify a bit the Procmail module/subroutine/recipe
        concept.

       Header file modules

        These are like #include .h files in C, they define some common
        variables, but do not contain actual code.

        o   pm-javar.rc -- Defines standard variables: SPC WSPC NSPC SPCL and
            perl styled \s \d \D \w \W and \a \A (alphabet only)
        o   headers.rc -- From Alan's procmail-lib. Define standard regexp
            and macros: address, from, to, cc, list_precedence

       General modules

        o   *pm-jafrom.rc* -- Derive FROM field without calling `formail'
            unnecessarily. If all fails, use formail.
        o   *get-from.rc* -- From Alan's procmail-lib. get the "best" from
            address. Sets FROM and FRIENDLY, the latter being the "friendly"
            user name sans any address.
        o   *pm-jaaddr.rc* -- Subroutine to extract various email components
            from INPUT. Like address=[e-mail], net=com, account=foo...
        o   *pm-jastore.rc* -- Subroutine for general mailbox delivery.
            Define MBOX as the folder where to drop
            message and this subroutine will store it appropriately.
            Supports single mboxes, ".gz" mbox files, directory files and
            MH folders with rcvstore.

       Date and time handling

        For these, you extract the date from somewhere first and then feed
        the string to some of these subroutines:

        o   *pm-jatime.rc* -- a lowlevel subroutine. Parse time "hh:mm:ss"
            from variable INPUT
        o   *pm-jadate1.rc* -- a lowlevel subroutine. Parse date
            "Tue, 31 Dec 1997 19:32:57" from variable INPUT
        o   *pm-jadate2.rc* -- a lowlevel subroutine. Parse ISO standard date
            "1997-11-01 19:32:57" from variable INPUT
        o   *pm-jadate3.rc* -- a lowlevel subroutine. Parse date
            Tue Nov 25 19:32:57 from variable INPUT
        o   *pm-jadate4.rc* -- Call shell command "date" once to construct RFC
            "Tue, 31 Dec 1997 19:32:57" and parse the YY MM HH and other
            values. You usually use this subroutine if you can't get the date
            anywhere else.

       Date and time handling

        You use these recipes to get the date directly from the message:

        o   *pm-jadate.rc* -- higher level recipe. Read date from message's
            headers: From_ Received, or call shell `date' if none succeeds.
        o   *date.rc* -- higher level recipe.
            From Alan's procmail-lib: parse date or from headers
            Resent-Date:, Date, and From

       Forwarding and account modules

        o   *pm-japop3.rc* -- Pop3 movemail implemented with procmail. You can
            send a "pop3" request to move your messages from account X to
            account Y. Each message is send separately. This recipe listens
            "pop3" requests.
        o   *pm-jafwd.rc* -- controlling Forward remotedly. You can change the
            forward address with a "control message" or turn on/off the
            forwarding with a "control message"
        o   *pm-japing.rc* -- Send short reply to subject containing word
            "ping" to show that the account is up and that email address is
            valid.
        o   *correct-addr.rc* -- From alan's procmail lib. To help forward mail
            from an OLD address to a NEW address, and do some mailing list
            mail management. This recipe file is intended to make it easy
            for users to forward their mail from their old address to a new
            address, and, at the same time, educate their correspondents
            about it by CC'ing them with the mail.

       Vacation modules

        o   *pm-javac.rc* -- A framework for your vacation replies. This
            recipe will handle the vacation cache and compose an initial
            reply; which you only need to fill in. (Like putting vacation
            message to the body)
        o   *ackmail.rc* -- From Alan's procmail lib. procmail rc to
            acknowledge mail (with either a  vacation message, or an
            acknowledgement)

       Message-id based modules

        o   *pm-jadup.rc* -- Handle duplicate messages by Message-Id.
            Store duplicate message to separate folder.
        o   *dupcheck.rc* -- From Alan's procmail-lib. If the current mail has
            a "Message-Id:" header, run the mail through "formail -D",
            causing duplicate messages to be dropped. Can use md5 cache.

       Cron modules

        o   *pm-jacron.rc* -- A framework for your daily cron tasks. This
            recipe contains all the needed checks to ensure that your
            includerc is called whenever a day changes. (Day change is
            subject to messages you receive). Your own cron includerc is
            run once a day.

       Backup modules

        o   *pm-jabup.rc* -- Save messages to backup directory and keep only N
            messages per day. Idea by John Gianni, packaged by Jari. Note:
            The implementation will always call shell for each message you
            receive; so using this module is not recommended if you get
            many messages per day. Instead, use the cron module to clean
            the messages' backup directory only once a day, and not everytime
            a message arrives.

       Confirmation modules

        o   *pm-jacookie.rc* -- Handle cookie (unique id) confirmations.
            Also known as Procmail authentication service (PAS).
            This simple procmail module will accept messages only from
            user's who have returned a "cookie" key. You can use this to
            to protect your mailing list from false "subscribe" messages
            or from getting mail from unknown people, typically spammers
            who won't send the cookie back to you to "validate" themselves.
            Uses subroutine pm-jacookie1.rc, which generates the unique
            cookie; CRC 32 by default.
        o   See also Michelle's confirmation module for SmartList

       File Servers

        o   *pm-jasrv.rc* -- A Mime Procmail file server (MPFS) It contains
            all the instructions and supports several MIME encoding types:
            text/plain and gzip. The keyword SEND is configurable. You
            can set up as many files servers as you need to different
            directories by changing the SEND keyword. MPFS supports
            password for file access.
        o   *commands.rc* -- From Alan's procmail-lib, check for commands
            in the subject line. Handles commands (send|get)
            [help|info|procmail info|procmail lib|procmailrc] and few
            others.
        o   *send-file.rc* is a very simplistic piece of procmail code
            to send file (non-MIME support) requested in subject line.
            http://www.universe.digex.net/~mbr/unix/send-file.html

       Mime modules

        o   *pm-jamime.rc* -- Subroutine to read MIME headers and put the
            mime version, boundary string, content-type information to
            variables.
        o   *pm-jamime-decode.rc* -- recipe to decode quoted-printable
            or base64 encoding in the body.
        o   *pm-jamime-kill.rc* -- Recipe for attachment killing: wipes out the
            extra mime cruft leaving only the plain text. Applications for
            killing: ms-tnef attachement (MS Explorer 7k),
            html attachements (netscape, MS Express) vcard (Netscape),
            PCX attachement (Lotus Notes).
        o   *pm-jamime-save.rc* -- Recipe for saving simple file attachement.
            When you receive _ONE_ file attachement in a message, this
            recipe can save it to separate directory. The content is
            also decoded (base64,qp) while saving to file.

       Filtering message body or headers

        o   *pm-jadaemon.rc* -- Handle DAEMON messages by changing subject to
            reflect the a) error reason b) to whom the message was originally
            sent c) original subject sent and what was the subject. Store the
            DAEMON messages to separate folder.
        o   *pm-jasubject.rc* -- Standardize Subject "Re[32]: FW: Sv: message"
            or any other derivate to de facto "Re: message"
        o   *pm-janetmind.rc* -- Reformat http://minder.netmind.com/ messages,
            The default 4k message is shortened to few important lines.

       Miscellaneus modules

        o   *pm-jaempty.rc* -- check if message body is empty (nothing
            relevant) Define variable BODY_EMPTY to "yes" or "no" if
            message is empty.
        o   *pm-janslookup.rc* -- Run nslookup on given address. If you
            compose return address with "formail -rt -x To:" you can
            verify if domain is registered before sending reply. Uses cache
            for already looked up domains.
        o   *guess-mua.rc* -- Guess the Mail User Agent and set MUA:
            MH,PINE,MAIL

       Mailing list modules

        o   *Microlist* a small mailing list by david hunt <[e-mail]>
            ...This version contains vars set for my environment and needs,
            and requires resetting of those vars before use. It's exact
            function and use will remain a mystery until I get a readme
            file written for it. If anyone wants to use it, I suggest you
            write to me first. If anyone has any suggestions or criticisms
            (no matter how harsh) please write
            http://www.west.net/~dh/homedir/microlist/microlist4.3
        o   *pm-jalist.rc* -- Subroutine to extract mailing list name from
            message. Do you need to add new recipe to your .procmailrc
            every time you subscribe to new mailing list? If you do,
            take a look at thsi module, which examines the message and
            defines variable `LIST' to hold the maling list name. You
            can use it directly to save the messages adaptively to
            correct folders. No more hand work and manual storing
            of mailing list messages.

    4.8 Where to get Procmail code and modules

       "Alan's procmail modules"
        Send subject "send procmail library" to Alan Stebbens
        <[e-mail]> http://reality.sgi.com/aks/

       "pm-code, Jari's Procmail modules"
        http://www.netforward.com/poboxes/?jari.aalto See pm-code.zip

       "Elijah's"
        http://www.qz.to/~eli/src/procmail/rc.master.html

       "Concordia scripts"
        http://alcor.concordia.ca/topics/email/auto/procmail/
        ...We provide sample sets of recipes to get you started. The great
        thing about the concordia scripts is the fact that they are
        designed to run from a central location and be called from a
        .procmailrc installed in the user's ~/home directory.
        <[e-mail]>

       "Meng on procmail"
        http://icg.resnet.upenn.edu/procmail/
        http://res2.resnet.upenn.edu/procmail/
        ...goes into exhaustive detail about how I manage my mailing lists

        "David's" David Hunt <[e-mail]>
        http://www.west.net/~dh/homedir/pmdir/
        ...My .procmailrc and .forward files can be viewed at

    4.9 Procmail code to filter UBE

        _Sysadms_ _remember_ : Spam filtering is much more efficiently done
        in the MTA, especially if you just looking at From and To lines.
        For example, I you can setup in Exim a rule that blocks \d.*@aol\.com
        (that is any aol.com local part that begins with a digit). AOL
        guarantees that _none_ of their addresses being with a digit. Exim
        rejects such bogus addresses at the SMTP level before the message
        is received.

       "Daniel's smap filter"
        1997-09-13 Daniel Smith <[e-mail]> sent excellent spam filter
        called `spamc.rc'. It used some nice heuristics and filters from
        various people, icluding [david] and [philip].
        Later Dan made substantial changes to it and the new version is
        available from ftp://ftp.bristol.nl/pub/users/DanS/spamcheck

       "pm-jaube.rc Jari's ube filter (compiled from others)"
        After Daniel Smith posted his spam recipes to procmail mailing
        list, Jari investigated them and compiled other recipes to a
        general purpose UBE module that needs no special setup and can be
        installed via simple INCLUDERC. No additional ube-list files are
        used, all UBE all detected happens using procmail rules. The module
        is included in kit `pm-code.zip'.

       "Catherine A. Hampton's Spambouncer"
        http://www.best.com/~ariel/nospam/
        ...The attached set of procmail recipes/filters, which I call
        The Spam Bouncer, are for users who are sick of spam (unsolicited
        junk mail email) and want to filter it out of their mail as easily
        as possible. These recipes can be used as shared recipes for a
        whole system, or by an individual for their own mailbox only.

       "Protect yourself from spam: A practical guide to procmail"
        http://www.sun.com/sunworldonline/swol-12-1997/swol-12-spam.html
        ...take you, step by step, through everything you need to know in
        order to enlist the aid of a Unix host in filtering unwanted e-mail
        traffic. This page is excellent to get you started with procmail
        and filtering with simple recipes and how to store messages to
        folders. Recommended for newcomers to Procmail.

       "Junkfilter" by Gregory Sutter <[e-mail]>
        http://www.pobox.com/~gsutter/junkfilter/
        ...Junkfilter is a user-configurable procmail-based filter system
        for electronic mail. Recipes include checks for forged headers,
        key words, common spam domains, relay servers and many others.

       "Download procmail spam filters"
        http://www.telebyte.com/stopspamr
        This is excellent site and contains many other spam stop pointers.

       "SpamDunk"
        http://www.interlog.com/~waltdnes
        http://www.interlog.com/~waltdnes/beta/techie.htm
        ...This webpage shows a commented example of a working .procmailrc
        file that works for me. I have tried to make things as generic as
        possible, but there are no guarantees that it will work for anyone
        else.