procmail: Timeout, terminating "/usr/bin/bogofilter"

[Daniel Leidert]

Am Sonntag, den 14.01.2007, 18:47 +0100 schrieb Matthias Andree:
> Daniel Leidert <daniel.leidert.spam at gmx.net> writes:
> 
> R> Hello,
> >
> > I went away a few days ago and back today I wanted to fetch all my mails
> > (ca. 10000 at once). My system consists of: fetchmail -> exim ->
> > procmail -> (several filters, bogofilter, spamassassin, clamassassin) ->
> > cyrus
> >
> > And today I found some weired behaviour. There were more and more
> > bogofilter processes - the amount increased up to 600-1000 in a really
> > short time (only 2000-3000 mails were downloaded), _but_ bogofilter
> >
> Is that heavy-duty server class hardware, with battery-backed RAID10 or
> RAID50 or stuff?

No. A simple system, not a high-end one. It's just m private
mail-server.

> If not, the usual hard disk thrashing will be enough to
> bring the box to a screaching halt, as you experienced.

Hm. Sounds logical. I just never observed this issue and I use my system
now for (IIRC) 2 years or more. So the behaviour was a surpsise.

> I'd say avoid running more than a few dozen bogofilter processes at the
> same time, the more you run, the more contention you'll experience

Logical. Because of this I asked for a build-in possibility to reduce
the number of running processes.

> particularly if you're using on-the-fly updating (-u).  I've never seen
> such things happen on Postfix, because I have tight limits on the number
> of local delivery processes.
> 
> Last time I checked, several years ago, Exim 3.34, it was not at all
> robust towards increasing load.

There is an queue_only_load option I never used. But for now I reduced
the amount in smtp_accept_queue_per_connection and removed the
auto-update switch of bogofilter (-u).

> Be sure to let Exim queue incoming messages rather than trying to
> deliver them all the way through at the same rate as fetchmail is
> feeding them.

The exim configuration I used defined, that everything after the first
100 messages is queued (only the first 100 are delivered immediately).
But yesterday the amount of bogofilter processes still increased to 450
for only 800 mails. So I'm not sure, if decreasing the amount in
smtp_accept_queue_per_connection will help. I will see.

@David: Just a feedback to your answers and suggestions: I got your
mails and already read them. I will send feedback in the next days.

Regards, Daniel