procmail: Timeout, terminating "/usr/bin/bogofilter"

[David Relson]

On Sun, 14 Jan 2007 02:31:01 +0100
Daniel Leidert wrote:

> Hello,
> 
> I went away a few days ago and back today I wanted to fetch all my
> mails (ca. 10000 at once). My system consists of: fetchmail -> exim ->
> procmail -> (several filters, bogofilter, spamassassin, clamassassin)
> -> cyrus
> 
> And today I found some weired behaviour. There were more and more
> bogofilter processes - the amount increased up to 600-1000 in a really
> short time (only 2000-3000 mails were downloaded), _but_ bogofilter
> worked (no errors about the environment or similar problems - so
> bogofilter definitely processed mails). Then procmail began to
> terminate bogofilter process by process showing the error in the
> subject:
> 
> procmail: Timeout, terminating "/usr/bin/bogofilter"
> 
> (no bogofilter return/error value message). I've checked
> my .bogofilter directory with bogoutil and bogoutil verified, that it
> is ok. These many processed "DoSed" my machine. It was almost
> unusable until everything finished (it filled my RAM and SWAP -
> >2GB!).
> 
> So what could cause this behaviour? My large database (> 200MB)? Or is
> this the intended design (I can remember times, where I had to fetch
> several 10000 mails at once and I cannot remember this behaviour)?
> Harddrives and Memory seem to be ok. I've checked my installation log
> but didn't find anything I had installed/updated, that seem to be
> related to this behaviour. Can I configure bogofilter to not start
> more than 10 processes? I didn't find anything in the FAQ or the docs.
> 
> Regards, Daniel

Hi Daniel,

I encountered something similar recently.  My environment is postfix,
procmail, and bogofilter.  At present postfix has a 200 MB file size
limit and wordlist.db was getting near that.  Bogofilter was detecting
the nearness.  Couple that with my use of "-U" (autoupdate) which
requires write access to the database.  The result was bogofilter
refusing to run.  This resulted in mail being delivered without
bogofilter scoring (as shown by lack of X-Bogosity line), multiple
copies of the messages (as procmail repeatedly tried running
bogofilter), and multiple instances of bogofilter running.  In this
situation, the fix was to run "bf_compact" to compact wordlist.db so
that the postfix file size limit wasn't being triggered.  Longer term,
the options seem to be increase the file size limit, stop using "-U" to
slow down database growth, or use bogoutil to discard old tokens.

Don't know if this will help any, though my symptoms are very similar
to yours!

HTH,

David