ProcmailMiniTutorial:
AutomatedMailHandling
byJimDennis,Proprietor,StarshineTechnicalServices
ConvertedtoHTMLbyHeatherStern
[Link]
ofbackgroundfortheintermediateUnixuseronhowtouseprocmail.
Asa"little"language(tousetheacademicterm)procmaillacksmanyofthefeaturesandconstructsoftraditional,
[Link]"while"or"for"[Link]"knows"alotaboutUnixmaildelivery
conventionsandfile/directorypermissionsandinparticularaboutfilelocking.
Althoughitispossibletowriteacustommailfilteringscriptinanyprogramminglanguageusingthefacilitiesinstalled
onmostUnixsystemswe'llshowthatprocmailisthetoolofchoiceamongsysadminsandadvancedUnixusers.
UnixmailsystemsconsistofMTA's(mailtransportagentslikesendmail,smail,qmailmmdfetc),MDA's(delivery
agentslikesendmail,deliver,andprocmail),andMUA's(useragentslikeelm,pine,/bin/mail,mh,Eudora,and
Pegasus).
[Link]
compatibleMTA'shavetheabilitytodispatchmail*through*acustomfilterorprogramthrougheitheroftwo
mechani[Link]
Thealiasesmechanismusesasinglefile(usually/etc/aliasesor/usr/lib/aliases)[Link]
[Link](asauser)can'tmodifyit.
The".forward"[Link]
[Link],afilename,oraprogram(filter).Usuallythefile*must*beownedbythe
userorrootand*mustnot*be"writeable"byotherusers(goodversionsofsendmailcheckthesefactorsforsecurity
reasons).
It'salsopossible,withsomeversionsofsendmail,foryoutospecifymultipleaddresses,programs,orfiles,separated
[Link]'llskipthedetailsofthat.
[Link]:
"|$HOME/bin/[Link]"
Notethequotesandthe"pipe"[Link].
"[Link]"couldbeaBourneshellscript,anawkorperlscript,acompiledCprogramoranyothersortoffilter
youwantedtowrite.
However"[Link]"wouldhavetobewrittentohandleaplethoraofdetailsabouthowsendmailwouldpassthe
messages(headersandbody)toit,howyouwouldreturnvaluestosendmail,howyou'dhandlefilelocking(incasemail
cameinwhile"[Link]"wasstillprocessingone,etc).
That'swhatprocmailgivesus.
WhatI'vediscussedsofaristhegeneralinformationthatappliestoallsendmailcompatibleMTA/MDA's.
So,[Link].(Thisissafetodo
beforeyoudoanyconfigurationofprocmailitselfassumingthatthepackage'sbinariesareinstalled).Here'sthe
canonicalexample,pastedfromtheprocmailmanpages:
"|IFS=''&&exec/usr/local/bin/procmailf||exit75#YOUR_USERNAME"
[Link]'sbecausemyexamplewasflawedfor
simplicity'ssake.
Whatthismessmeanstosendmail(paraphrasingintoEnglish)is:
Pipethemailtothefollowingcommand(s):
Settheshell's"interfieldseperator"(IFS)toaspace,andifthatwentO.K.(&&)executetheprogramnamed
"/usr/local/bin/procmail"
(yoursmayneedtobedifferenttrythecommand'whichprocmail'toseeifit'sonthepathor'locateprocmail'
ifyoursystemmaintainsafilelocatordatabase).
Theprocmailprogramisbeingpassedasetofswitches:"f"whichtellsitto"updatetimestampintheleading
the'From'lineintheheader"
(thislastbitisratherobscureandhastodowithhowmessagesarenormallystoredinyour"incoming"ormailfile
or"spool"asweUnixhacksliketocallit).
[Link]'s"||"operatorwhichisbasicallyacontinuationfrom
the"and"(&&)[Link]"or"(ifthatcommanddidn'[Link])
then"exit"(stopprocessing)andreturnanerrornumber75(whichwepresumeismeaningfultosendmailthe
programthatcalledthiscommand).
[Link]:
"isnotactuallyaparameterthatisrequiredbyprocmail,infact,itwillbediscardedbyshbefore
procmaileverseesititishoweveranecessarykludgeagainstoveroptimisingsendmailprograms:"
YoushouldjustchangethephraseYOUR_NAMEtoyourloginnameonthatsystem.
[Link],minimallyeditedandforgotten.
[Link]
file(.procmailrc)[Link]
appendnewmessagesintoyournormalspoolfile.
[Link]
oryoucanuseitanyway.
[Link]
couldactuallycallthisfileanythingyouwantedbutthenyou'[Link]
(rightbeforethe"||"operator).Almosteveryonejustusesthedefault.
[Link]'vetalkedaboutithoweverythinggetsroutedtoprocmailwhich
mostlyinvolvessendmailandtheBourneshell'[Link]'sareconfiguredtouse/bin/sh(theBourne
shell)[Link]"pipes."
So,here'[Link]:
:0c:
$HOME/[Link]
Thisjustappendsanextracopyofallincomingmailtoafilenamed"[Link]"inyourhomedirectory.
[Link]'sbeensuggestedthatyoushouldexplicity
setSHELL=/bin/sh(ortheclosestderivativetoBourneShellavailableonyoursystem).I'veneverhadtoworryaboutthat
sincetheshellsIuseonmostsystemsarealreadyBournecompatible.
However,cshandothershellusersshouldtakenotethatalloftheprocmailrecipeexamplesthatI'veeverseenuse
Bournesyntax.
The:0linemarksthebeginningofa"recipe"(procedure,clause,whatever.:0canbefollowedbeanyofanumberof
"flags."[Link]'reusinginthisexampleis
'c'for"copy."
Youmightaskwhytherecipestartswitha:[Link]:x(wherexwasanumber).Thiswasahintto
[Link],theoptionwasaddedtoprecedeconditionswitha
leadingasterisksotheydidn'thavetobemanuallycounted.:0thencametomeansomethinglike:"countthem
yourself."
[Link]
givenprocmailwillpickoneautomatically.
[Link]
processingthelastmessageyou'[Link]'t
[Link]
jumbledinunpredictableways(theresultwillnotbeaproperlyformattedmailfolder).
[Link]'tcarewhatthe
nameofthelockfilewouldbe(sincewe'renotgoingtohave*other*programswritingintothebackupfile).Sowe
leavethelastfield(afterthecolon)[Link].
Ifweleavethe:offoftherecipeheaderline(ommittingthelastfieldentirely)thennolockfileisused.
Thisisappropriatewheneverweintendtoonlyreadfromthefilesintherecipeorincaseswhereweintendtoonly
writeshort,singlelineentriestoafileinnoparticularorder(likelogfileentries).
Thewayprocmailworksis:
Itreceivesasinglemessagefromsendmail(orsomesendmailcompatibleMTA/MDA).Theremaybeseveralprocmail
processingrunningcurrentlysincenewmessagesmaybecominginfasterthantheyarebeingprocessed.
Itopensitsrecipefile(.procmailrcbydefaultorspecifiedonitscommandline)andparseseachrecipefromthefirstto
thelastuntilamessagehasbeen"delivered"(or"disposedof"asthecasemaybe).
Anyrecipecanbea"disposition"or"delivery"[Link]"delivered"thenprocmailcloses
itsfiles,removesitslocksandexits.
Ifprocmailreachestheendofit'srcfile(andthusalloftheINCLUDE'dfiles)without"disposing"ofthemessage
thanthemessageisappendedtoyourspoolfile(whichlookslikeanormaldeliverytoyouandallofyour"mailuser
agents"likeEudora,elm,etc).
Thisexplainswhyprocmailissoforgivingifyouhave*no*.[Link]
becauseithasreachedtheendofallitsrecipes(therewerenone).
The'c'flagcausesarecipetoworkona"copy"ofthemessagemeaningthatanyactionstakenbythatrecipearenot
consideredtobe"dispositions"ofthemessage.
Withoutthe'c'flagthisrecipewouldcatchallincomingmessages,[Link]
ofitwouldgetintoyourspoolfileandnoneoftheotherrecipeswouldbeparsed.
[Link]'[Link]
[Link],forwardittosomeothermailaddress,orfilterit
throughaprogram.
Actuallythereisonespecialformof"delivery"or"disposition"[Link]
name(ratherthanafilename)[Link]
basedonseveralrathercomplicatedfactorsthatyoudon'thavetoworryaboutunlessyouusetheRandMHsystem,or
someotherrelativelyobscureand"exotic"mailagent.
Aprocmailrecipegenerallyconsistsofthreepartsastartline(:0withsomeflags)someconditions(linesstartingwith
a'*'asteriskcharacter)andone"delivery"linewhichcanbefile/directorynameoralinestartingwitha'!'bang
characterora'|'pipecharacter.
Here'sanotherexample:
:0
*^From.*[Link]@[Link]
/dev/null
Thisisasimpleoneconsistingofnoflags,[Link]
"someoneIdon'tlike."(/dev/nullunderUnixisa"bitbucket"abottomlesswellfortossingunwantedoutputDOS
hasasimilarconceptbutit'snotnearlyashandy).
Here'samorecomplexone:
:0
*!^FROM_DAEMON
*!^FROM_MAILER
*!^XLoop:myaddress@[Link]
|$HOME/bin/[Link]
Thisconsistsofasetofnegativeconditions(noticethattheconditionsallstartwiththe'!'character).Thismeans:for
anymailthatdidn'tcomefroma"daemon"(someautomatedprocess)anddidn'tcomea"mailer"(someotherautomated
process)andwhichdoesn'tcontainanyheaderlineoftheform:"XLoop:myadd..."senditthroughthescriptinmybin
directory.
Icanputthescriptdirectlyinthercfile(whichiswhatmostprocmailusersdomostofthetime).Thisscriptmightdo
[Link]
mailtobedeliveredandanyrecipesafterthiswillonlybereachedbymailfromDAEMONs,MAILERsandanymailwiththat
particularXLoop:lineintheheader.
ThesetwoparticularFROM_conditionsareactually"special."Theyarepresetbyprocmailandactuallyrefertoacoupleof
rathercomplicatedregularexpressionsthataretailoredtomatchthesortsofthingsthatarefoundintheheadersofmost
mailfromdaemonsandmailers.
TheXLoop:lineisanormalprocmailcondition.IntheRFC822document(whichdefineswhatemailheadersshould
looklikeontheInternet)anylinestartedwithXisa"custom"[Link]
canaddprettymuchanyXlineitwants.
AcommonprocmailidiomistoaddanXLoop:linetotheheaderofanymessagethatwesendoutandtocheckfor
ourownXLoop:[Link]"mailloops"situationswhereourmail
getsforwardedor"bounced"backtousandweendlesslyrespondtoit.
So,here'[Link]
withtherecipeheader.
:0
...thenweaddouronecondition(thatthemailappearstobefromthepersoninquestion):
*^FROMharasser@[Link]
FROMisa"magic"valueforprocmailitchecksfrom,resentby,[Link]^From:
whichwouldonlymatchtheheaderline(s)thatstartwiththestring"From:"
The^(hiccupor,moretechnically"caret")isa"regularexpressionanchor"(atechiephrasethatmeans"itspecifies
*where*thepatternmustbefoundinordertomatch."Thereisawholebookonregularexpression(O'Reilly&
Associates)."regexes"permeatemanyUnixutilities,scriptinglanguages,[Link]
differencesin"regex"[Link]'grep'or'egrep'isanexcellentplaceto
learnmore.
Inthiscasethehiccupmeansthatthepatternmustoccuratthebeginningofaline(whichisitsusualmeaningingrep,
ed/sed,awk,andothercontexts).
...andweaddacoupleofconditionstoavoidloopingandtoavoidrespondingtoautomatedsystems
*!^FROM_DAEMON
*!^FROM_MAILER
(Theseareacouplemore"magic"[Link]
ifyou'recuriousorneedtotweakaspecialconditionthatissimilartooneortheotherofthese).
...andonemoretopreventsometrickyloop:
*!^XLoop:myaddress@[Link]
(Allofthesepatternsstartwith"bangs"(exclammationpoints)becausetheconditionisthat*no*lineoftheheaderstart
[Link]'bang'inthiscase(andmostotherregexcontexts)"negates"or"reverses"themeaningof
thepattern).
...nowweadda"disposition"theautoresponse.
|(formailrk\
A"XLoop:yourname@[Link]"\
A"Precendence:junk";\
echo"Pleasedon'tsendmeanymoremail";\
echo"Thisisanautomatedresponse";\
echo"I'llneverseeyourmessage";\
echo"So,GOAWAY")|$SENDMAILtoi
Thisisprettycomplicatedbuthere'showitworks:
Thepipecharactertellsprocmailthatitshouldlaunchaprogramandfeedthemessagetoit.
TheopenparenthesisisaBourneshellconstructthatgroupsasetofcommandsinsuchawayastocombinethe
outputfromallofthemintoone"stream."We'llexplainthismorelater.
The'formail'[Link]"formats"mailheaders
accordingtoitscommandlineswitchesanditsinput.
rktells'formail'toformata"reply"andto"keep"[Link]
headerandbodyasinput.
TheAparameterstellsformailto"add"[Link]
theAswitchmustbeenclosedinquotessotheshelltreatsthewholestring(spacesandall)assingle
parameters.
[Link],allof
thelinesendinginbackslashesarepassedtotheshellasonelongline.
This"trailingbackslash"or"linecontinuation"characterisacommonUnixidiomfoundinanumberof
programminglanguagesandconfigurationfileformats.
Thesemicolonstelltheshelltoexecuteanothercommandtheyallowseveralcommandstobeissuedon
thesamecommandline.
[Link]'cat'command
[Link]'fortune'or'date'and
theiroutputwouldbecombinedwiththerestofthis).
[Link]
outputfromallofthoseisfedintothenextpipewhichstartsthelocalcopyofsendmail(notethatthisis
anothervariablethatprocmailtoughtfullypresetsforus).
Thetswitchonsendmailtellittotakethe"To:"addressfromtheheaderofit'sinput(where'formailr'
putit)andtheoiswitchenablesthesendmail"option"to"ignore"linesthatconsistonlyofa'dot'(don't
worryaboutthedetailsonthat).
[Link]
expressions(thosewierdthingsonthe'*'conditionallines)andshellquotingandcommandsyntax,andhowtoformat
areplyheaderthatwillbeacceptabletosendmail(the'formail'and'sendmail'stuff)arethepartsthatrequiresomuch
explanation.
ThebestinfoonmailbotsthatI'vefoundusedtobemaintainedbyNancyMcGough(sp??)attheInfiniteInkwebpages:
[Link]
MoreinformationaboutprocmailcanbefoundinEraEriksson's"MiniFAQ."at[Link]
[Link]
IalsohaveafewprocmailandSmartListlinksoffofmyownwebpages.
