Bug#293207: bogofilter: last two versions caused db errors

[Karl Schmidt]

Matthias Andree wrote:

>>
>>I have:
>>
>>libdb4.3       4.3.27-1
> 
> 
> Please run "bogofilter -V" to check the bogofilter and Berkeley DB
> versions, the first two lines are sufficient. Do this with either
> bogofilter version. Remember that if you're inadvertently going forth
> and back between Berkeley DB versions, your database environment may
> break like this. Going backwards isn't supported (so bogoutil -d before
> the upgrade, remove the database, downgrade, bogoutil -l), going
> forwards requires you to remove the environment _BEFORE_ the update.
> 
> I have rewritten parts of README.db after the 0.93.5 release, hence I'm
> adding the rewritten version below, perhaps it can help.
> 
> 
Installing bogofilter on a Debian testing box gives us:

ii  bogofilter     0.93.5-1       a fast Bayesian spam filter

$ bogofilter -V
bogofilter version 0.93.5
     Database: Sleepycat Software: Berkeley DB 4.3.27: (December 22, 2004

I delete all the files in the db directory and run the following script 
(as I've had to rebuild a few times now<g>):

#!/bin/bash
bogofilter -M -s -d /etc/bogofilter -I /home/karl/mail/zs-archived-spam2004
bogofilter -M -s -d /etc/bogofilter -I /home/karl/mail/zs-archived-spam2003
bogofilter -M -s -d /etc/bogofilter -I /home/karl/mail/s-archived-spam
bogofilter -M -n -d /etc/bogofilter -I /home/karl/mail/z-archived2004
bogofilter -M -n -d /etc/bogofilter -I /home/karl/mail/archived
bogofilter -M -n -d /etc/bogofilter -I /home/karl/mail/list-servers/EXIM
bogofilter -M -n -d /etc/bogofilter -I
chown  Debian-exim.Debian-exim /etc/bogofilter/*

Everything works (not sure if it is tagging quite as much spam) then it 
ends ups stopping after about 48 hours.

This is on a Tyan MB with ECC memory, antec powersupply - I think a 
quite stable system running bind, dhcp,hylasfax, samba, nfs, imap all 
flawlessly. I would suspect falky hardware at this point except going 
back to the older version fixes things.

Only other thing I can suspect is that exim is threaded - could there be 
a locking problem I'm seeing running two requests at a time? I can 
imagine that 48 hours would be long enough to be filtering two messages 
at the same time. That would explain why most people running in a single 
thread POP service manner would not see this bug.

The basic fact is I am sure I recreated the databases and didn't upgrade 
and try to run the old data base  (which if I remember would have failed 
at once.) Going back to the old version and once again reproducing the 
databases fixes the problem.

I can think that it would be easy to test by running two or three 
instances of bogofilter at the same time on some mail files. One can 
write a script that will fork and you might want to add it to your 
testing procedure. Hope this helps.

I hope I didn't sound off base here and hope I haven't ruffled any 
feathers, but I really do think that these should spend some time in 
unstable.

----------------------------------------------------------------
Karl Schmidt                         EMail Karl at xtronics.com
Transtronics, Inc.    		       WEB http://xtronics.com
3209 West 9th Street                    Ph (785) 841-3089
Lawrence, KS 66049                     FAX (785) 841-0434


A patent provides one a license to enrich his lawyer. -kps
----------------------------------------------------------------