bf_compact

[David Relson]

On Sat, 14 May 2005 22:57:11 +0100
R Kimber wrote:

> On Sat, 14 May 2005 20:18:54 +0200
> Matthias Andree <matthias.andree at gmx.de> wrote:
> 
> > > Yes.  Thanks.  I wasn't offended - just that I used it as advertised
> > > and my wordlist and config files disappeared.
> > 
> > If your *wordlist* disappeared, then that is a genuine bug that needs
> > to be fixed - in that case, have my apologies for mis-reading your
> > previous mail. Configuration files in .bogofilter? DB_CONFIG perhaps,
> > we are copying that if present, but no other configuration files
> > should be in that directory, no?
> 
> /etc/bogofilter.cf suggests one can have:-
> user_config_file=~/.bogofilter/config
> which is what I had.

Good point.  bogofilter.cf definitely mentions putting the config file
in ~/.bogofilter.  One solution would be to add lines like the
following to bf_compact:

   FILES=`ls $BOGOHOME/* | egrep -v "$BOGOHOME/(log.0|__db.|lockfile.)"`
   cp -p $FILES $TEMP

> More accurately, for the record, what happened was that my .bogofilter
> directory contained a config file, a wordlist.db, and some log files
> etc left in place from previously running in tansactional mode, plus a
> copy of a previous wordlist in a subdirectory.
> 
> After running first
> 
> bf_compact .
> 
> and then (after having got mv: cannot move `.' to `..old': Device or
> resource busy) I ran
> 
> bf_compact ~/.bogofilter wordlist.db
> 
> I was left with an empty wordlist.db (reduced from about
> 20MB+ to about 8K) in .bogofilter and also in .bogofilter.old, and all
> the other files in .bogofilter were deleted. including the config file,
> the log files, and the subdirectory.

Attached are revised copies of bf_compact.in (for 0.94.10 or newer) and
bf_compact.  The script now copies all files in $BOGOHOME, except
database and database environment files.  AFAICT both the following
work fine:

  bf_compact dir
  bf_compact dir wordlist.db

Note that if you have _more_ than 1 wordlist in dir you should use the
first form (which will compact all the wordlists) or else you should
list all the wordlists in the second form.  If dir has 2 wordlists and
you use the second form, the new directory will only have the one
wordlist named in the command.  

Honestly, I think the bf_compact is getting much too long and
complicated.  I wish that matters were simpler, but they just ain't :-<

> > > It might still be better for those that use it if bf_compact got its
> > > transactional info from the config file, though, as there might be
> > > old unused log files around from a previous transactional usage - or
> > > wouldn't that matter?
> > 
> > That wouldn't work reliably. I don't need to configure transactions, I
> > can also just create the wordlist (i. e. the very first registration
> > of spam or ham) in transactional mode and bogofilter will remember
> > this, but couldn't tell from the configuration that you were using
> > transactions.
> 
> I just thought that because bf_compact seems to check whether it's
> dealing with a transactional mode by looking for log files, if it
> detects old log files it might be doing something to a non-transactional
> setup that is best not done. But David has said this is not so, so
> that's fine.
> 
> If it's any help, I could try to write an alternative script using the
> default directory structure, but I guess if it's not necessary to
> compact the db anyway, there's no point.  And it's not as if I have a
> slow machine :)

AFAICT there's little real need for most people to compact the
database.  Compacting it once in a while is probably file.  By "once in
a while", I'm thinking twice a year, or when it gets over 100MB, or
some such thing.

HTH,

David