DB corruption within minutes

[Matthias Andree]

Gyepi SAM <gyepi at praxis-sw.com> writes:

>> Could bogofilter use an external lock file of some sort?  Would this help 
>> or would it create even more trouble and complication?
>
> Sure, if the problem is with locking the db files.

OK, I have 16 hours of serialized bogofilter now, with flock() in my
.mailfilter surrounding it, that means I have at most one bogofilter
running at a time. I has gone without corruption, with near 900 mails
passing through it, which formerly would have killed the db in less than
two hours.

I have a strong suspect: our db locking. It always hits my goodlist.db,
never my spamlist.db (but then again, there's still more ham than spam
here), and bogofilter running in -u)pdate mode.

I have added t.lock2 and made minor fixes to bogofilter.

I'm wondering if we can do things the way we do. We release the lock
before closing the data base, which does not look right. BDB has
internal caches (like stdio) that need to be flushed by ->close before
we can let go of the lock, so the ordering would likely have to be
db_close, db_lock_release, but db_close kills the handle, so all of this
should be more tightly integrated.

Plus, I believe we cannot release the lock, have someone else update the
db and then grab the lock again to proceed. The pages may have changed,
so we have inconsistent cache/disk data.

-- 
Matthias Andree