database environment [was: bogofilter 0.93.4 - new current release]

[Chris Wilkes]

On Sun, Jan 09, 2005 at 12:49:37PM -0500, David Relson wrote:
> Hi Chris,
> 
> Sorry to hear you're encountering difficulties.  Let's see if we can
> figure what's going on...
> 
> On Sun, 9 Jan 2005 09:30:02 -0800
> Chris Wilkes wrote:
> 
> > On Sun, Jan 09, 2005 at 10:38:57AM -0500, David Relson wrote:
> > > 
> > > The 0.93.4 includes additional tweaks for supporting Berkeley DB
> > > transactions, new options for ./configure for russian language support
> > > (specifically "--enable-russian" and "--default-charset=charsetname"),
> > > and a common "long options" file shared by bogofilter, bogolexer, and
> > > bogoutil.  The usual number of small changes is also included.  Read
> > > below for further details.
> > 
> > I suffered from the same lock table errors reported by others in 0.93.3
> > when trying to do a bf_compact.  I updated to 0.93.4 and tried again:
> > 
> > $ bf_compact $BOGOFILTER_DIR wordlist.db
> >   mv: cannot overwrite directory `/home/vmail/maildirs/cwilkes/.bogofilter/.old/.bogofilter'
> > $ rmdir /home/vmail/maildirs/cwilkes/.bogofilter/.old/.bogofilter
> > $ bf_compact $BOGOFILTER_DIR wordlist.db
> > mv: cannot move `/home/vmail/maildirs/cwilkes/.bogofilter/' to a
> > subdirectory of itself, `/home/vmail/maildirs/cwilkes/.bogofilter/.old/.bogofilter'
> 
> A "mv" failure sounds like permission problems with the parent
> directory.  What does 
> 
>   ls -la /home/vmail/maildirs/cwilkes/.bogofilter/.old
> 
> report?

I had to create that directory for the first time around.  I would try
and do it again but I'm getting the following errors:

  $ bf_compact $BOGOFILTER_DIR wordlist.db
  /home/vmail/maildirs/cwilkes/.bogofilter/wordlist.db.new: No such file or directory

which keeps on repeating.

> > Can I use the wordlist.df found in bf_compact.$$/ ?
> 
> Not by itself.  A Berkeley DB environment includes _all_ the files in
> the directory.  As long as _NO_ bogofilter process is running, you can
> cheat and use "cp -f bf_compact.$$/* $BOGOFILTER_DIR".

Just did that -- what should I do with all the log.* files?

> > I was also able to do a
> >    bogoutil -d wordlist.db | bogoutil -l wordlist.db.new
> > without the lock problem.  However I sometimes get these errors:
> >   $ mv wordlist.db.new wordlist.db
> >   $ bogofilter -B $files -v
> >   /home/vmail/maildirs/cwilkes/.bogofilter/wordlist.db.new: No such file or directory
> > which is a little odd -- a simple grep for "new" in the $BOGOFILTER_DIR
> > doesn't show anything referencing the new wordlist file.

I take that back, but I'm not sure how to read the log.* file:
  $ grep --text \.new log.0000000062 | strings
  wordlist.db.new
  wordlist.db.new
  wordlist.db

> That's an odd one.  Bogofilter's DEBUG macros (activated by "-x ...")
> should help us.  What's the output of command:
> 
>    echo this is a test | bogofilter -x d -vv -C 

$ echo this is a test | bogofilter -x d -vv -C
open_lockfile:
open(/home/vmail/maildirs/cwilkes/.bogofilter//lockfile-p) succeeded, fd
#3
db_env_create: 0x8b85164
DB_ENV->set_cachesize(4)
DB_ENV->set_lk_max_locks(0x8b857e0, 16384)
DB_ENV->set_lk_max_objects(0x8b857e0, 16384)
DB_ENV->set_lk_detect(DB_LOCK_DEFAULT)
DB_ENV->set_lg_max(1048576)
DB_ENV->open(home=/home/vmail/maildirs/cwilkes/.bogofilter/)
db_version: Header version 4.2, library version 4.2
[pid 7150] DB->open(db=0x8b86118, file=wordlist.db, database=NIL,
type=1, flags=0x1000010=DB_RDONLY DB_AUTO_COMMIT , mode=0664) -> 0
Successful return: 0
DB->get_byteswapped: false
DB->fd: 5
DB->stat success, pagesize: 4096
X-Bogosity: Ham, tests=bogofilter, spamicity=0.025192, version=0.93.4
   int  cnt   prob  spamicity histogram
  0.00    1 0.025192 0.025192 #
  0.10    0 0.000000 0.025192
  0.20    0 0.000000 0.025192
  0.30    0 0.000000 0.025192
  0.40    0 0.000000 0.025192
  0.50    0 0.000000 0.025192
  0.60    0 0.000000 0.025192
  0.70    0 0.000000 0.025192
  0.80    0 0.000000 0.025192
  0.90    0 0.000000 0.025192
DB->close(/home/vmail/maildirs/cwilkes/.bogofilter/wordlist.db, nosync)
DB_ENV->close(0x8b857e0): Successful return: 0
close_lockfile