possible message count corruption?

[Jeremy Blosser]

Switching to the -dev list...

On Sep 19, Adrian Otto [aotto at aotto.com] wrote:
> Jeremy,
> > As mentioned previously, we're running some basic stress tests using
> > bogofilter to classify our entire incoming mail load as spam, and
> > something  interesting happened.  Note that we're using 0.7.3, so
> > it's possible this is known and fixed; we're looking seriously at
> > deploying this in production within weeks, so we grabbed the latest
> > version that was not explicitly tagged 'beta' on sf.net.
> 
> You should start using 0.7.4 today. I will be releasing it from beta this
> ...

Ok.

> Please see my comments below about the problem you are seeing...
> 
> > Anyway.  For each message we're running:
> > bogofilter -v >> bogofilter.log; bogofilter -s -v >> bogofilter.log
> > This ran fine for most of yesterday, but then last night I was glancing at
> > the log and saw this:
> >
> > % grep '^bogofilter:' bogofilter-log
> > ...
> > bogofilter: 5298 messages on the spam list
> > bogofilter: 1 messages on the spam list
> > ...
> 
> As far as I know, this is a new discovery. Is this something that you are
> able to reproduce? If so, please post a bug on Sourceforge with steps to

It has happened again, several times, as my other mail noted.  I'm not sure
how to reproduce it manually since this is just what's happening in the
regular course of piping our regular mail load through it.

> reproduce the problem. This will be useful in addition to the raw log. Also,
> set up an entry in syslog.conf to trap *.debug messages in a file. The lock
> and unlock messages will be interesting. I just noticed that I'm missing a
> call to openlog, so those messages are going unlabeled. I'll get a patch
> into the next version for that.

Logging all of our *.debug stuff on those servers is not an option due to
load.  I went ahead and patched our local copy of 0.7.4 to openlog() on
local6.  I also adding some basic timestamping to the log bogofilter is
writing when invoked by the MTA.

> You can upload your log file to http://aotto.com/incoming and I'll help
> coordinate a solution once we figure out why this happens.

Well, between switching to 0.7.4 and adding the above logging, I'll wait
til I see it happen again and then provide the more complete logs.