What does Bogofilter gain from Berkeley DB?

[David Relson]

On Sat, 11 Jun 2005 08:15:41 +1000
Ben Finney wrote:

> On 10-Jun-2005, Clint Adams wrote:
> > > target binary packages for the various non-default backends.
> > > (Debian is currently in this boat.)
> > 
> > Actually, that will change just as soon as the ftpmasters approve
> > the 0.94.14 packages.
> 
> What has changed in the 0.94.14 packages in regard to backend
> database?
> 
> > >   - use a default database backend that is less fragile?
> > 
> > I have no problems with BDB and bogofilter.
> 
> You are not alone. However, from the steady stream of cries for help
> on this list that are met with "yeah, that's because the Berkeley DB
> files got corrupted, you'll probably have to rebuild", I can only
> conclude that Berkeley DB is too fragile for a default configuration
> of bogofilter.

BerkeleyDB is fast and widely supported.  That was sufficient reason
for it to replace Judy, bogofilter's first database.  Admins have
learned to backup periodically in case of problems.

Some months ago transaction support was added to bogofilter.  It runs a
bit slower and uses logfiles to ensure safety, as far as possible.
Read file README.db if you want to learn more.  Bogofilter's default
built is bimodal - you can decide at run-time whether you want
transaction mode or non-transaction mode.

If you look further into bogofilter's mailing list archives, you'll
find that more people prefer to have transactions off than on, hence
the mode is "off" by default.  As mentioned it can be enabled/disabled
at run-time (either through command line or config file options).

Another option is to configure/build for sqlite which supports
transactions without the Berkeley DB overhead issues.  Sqlite _is_
somewhat slower, which may be a factor for some sites.

FWIW, in the course of almost 3 yrs with 500,000+ messages scored by
bogofilter and generally using '-u' and or '-uU' options, my mail
server has not had any database corruption problems.  Of course others
have not been so lucky.