new addition to FAQ

[David Relson]

On Tue, 7 Sep 2004 21:39:28 -0700
john sachs wrote:

> i use bogofilter with -u all the time and now i am getting these in my
> system log:
> 
> Sep  7 21:37:42 bogofilter[24852]: datastore_db.c:223: warning: data
> base file size approaches resource limit. Sep  7 21:37:42
> bogofilter[24852]: datastore_db.c:224:          write errors (bumping
> into the limit) can cause Sep  7 21:37:42 bogofilter[24852]:
> datastore_db.c:225:          data base corruption.
> 
> i dont know, but i guess my only option is to not use -u all the time
> and only do updates when things fail.  perhaps the real answer to this
> problem should go in the FAQ? thanks.
> -j

Hi John,

The needed info is in the FAQ, though it may not be readily obvious. 
Look for the sections about compacting the database and about
mailbox_size_limit (assuming you're using postfix).

I've been running "-u" since it was implemented in 2002.  In January
2004 I started thinking about all the "easy" messages, i.e. the spam
with score 1.000000 and the ham with score 0.000000, and how (maybe)
bogofilter didn't really need to be trained on them and how it would
slow the growth of the wordlist.  Using "thresh_update=0.01" in the
configfile causes bogofilter to _not_ autoupdate if the score is less
than 0.01 or above 0.99.  With that change in place, my site's wordlist
is now growing by 5 or 10 messages a day, rather than 1000+.  It's
helped slow the growth a lot and accuracy remains excellent (approx 2 FN
per 1000 spam).

I'll see about adding another entry to make the info easier to find.

By the way, googling for "data base file size approaches resource limit"
finds several bogofilter hits.  Unfortunately they point to the source
code or non-english, i.e. japanese and polish.

David

P.S.  I've CC'd this to the user's mailing list to help future google
searches.