DB corruption within minutes

[David Relson]

At 04:40 PM 1/6/03, Gyepi SAM wrote:

>On Mon, Jan 06, 2003 at 05:52:42PM +0100, Matthias Andree wrote:
... [snip] ...

>In reviewing the BDB documentation, they do point out that the only way to 
>avoid
>data corruption is to:
>
>1. Make a copy of the database, change it, then replace the original with 
>the copy
>2. Use transactions.

What's the real problem here?  That BerkeleyDB doesn't have real locks 
unless transactions are used?  That Linux has a lock problem?  Something else?

Could bogofilter use an external lock file of some sort?  Would this help 
or would it create even more trouble and complication?

>Option 1 is just plain stupid, so we're left with 2.

Given that we are updating/registering multiple words at a time, 
transactions might not be a bad way to go.  We won't need a transaction per 
word, just a transaction per message.  The overhead seems reasonablee.

>I have reviewed the code and see nothing obviously wrong, but I'll take 
>another look
>as soon as I am done with the mime stuff. The first step may be to add 
>more debugging messages.
>
>-Gyepi
>
>---------------------------------------------------------------------
>FAQ: http://bogofilter.sourceforge.net/bogofilter-faq.html
>To unsubscribe, e-mail: bogofilter-dev-unsubscribe at aotto.com
>For summary digest subscription: bogofilter-dev-digest-subscribe at aotto.com
>For more commands, e-mail: bogofilter-dev-help at aotto.com