bogoupgrade

[Rodrigo Bernardo Pimentel]

On Fri, Aug 22 2003 at 02:10:03PM BRT, David Relson <relson at osagesoftware.com> wrote:
> On Fri, 22 Aug 2003 13:37:02 -0300
> Rodrigo Bernardo Pimentel <rbp at isnomore.net> wrote:
> 
> ...[snip]...
> 
> >         BTW, why are bogofilter databases so often corrupted? 
(...)

> You ask some good questions, and I wish I knew the answer.  I'd like to
see the problem identified!!

        So I believe would everyone. Bogofilter has been a *great* help to
me (keeps me away from more than a hundred junk messages a day), I'll do
what I can to help :)

> What's your environment - distribution, kernel, BerkeleyDB version?  Is
there anything you're aware of that may be different about your setup?

        I use Debian GNU/Linux in a mixed testing/unstable state, kernel
2.4.20 (Debian package). I am now using bogofilter 0.14.3-1 (which requires
libdb4.1), but the problem also happened with different versions
(unfortunately, I can't really remember what they were).

        There's really nothing very unusual I can think about my
setup. Well, I do have a few versions of libdb installed (Debian allows
that, and even requires that, in unstable or testing, as different packages
use differente versions). But each package is compiles against a specific
version, so I think the problem is not there.

        I check messages via fetchmail, through procmail, and pipe them to
"/usr/bin/bogofilter -u -e -p"

> I just checked my 7 daily wordlist.db backups and db_verify thinks they're
all fine.  Of course, the fact that it's working for me doesn't mean a whole
lot.  FWIW, my wordlist.db is approx 44MB and has about 780,000 tokens in
it. 

        I'm now retraining with ~19.500 spam messages and ~17.500 ham
messages. My previous (now corrupt) wordlist.db is 42M (but was trained with
only about 7k spam messages).

        As I said, I'm going to backup and verify my wordlist.db daily,
perhaps that'll help locate the source of this recurrent corruption. I'll
also try and take the time to become familiar with the bogofilter code, so I
can be more helpful.

        Thanks a lot :)



                rbp
-- 
 Rodrigo Bernardo Pimentel                         <rbp at isnomore.net>
 http://isnomore.net
 GPG: <0x81F85A48>  7E62 9CA2 C95B FC86 B334 203E C011 2E4D 81F8 5A48 

Emotions are alien to me.  I'm a scientist.
          -- Spock, "This Side of Paradise", stardate 3417.3