[cvs] bogofilter/src globals.c,1.34,1.35 globals.h,1.30,1.31 lexer_v3.l,1.73,1.74 collect.c,1.14,1.15 bogolexer.c,1.39,1.40

[Matthias Andree]

On Sat, 06 Sep 2003, David Relson wrote:

> The question I'm addressing is:  What can we do to prevent bogofilter
> aborting when the flex code detects a fatal error?

Nothing. I wouldn't even want bogofilter to prevent such things.

We should make sure it shuts down its data base access gracefully (when
writing to it, or to clear DB-internal locks) and then terminate with an
error code.

> It's a problem whenever bogofilter exits with an error code because
> procmail (or whatever) will keep rerunning bogofilter which will keep
> getting the error.

Correct. As other mail will still be forwarded, through bogofilter, I
see no problem, I've seen such conditions more than once, running CVS
versions after all: the particular mail remains in the queue and clears
when bogofilter is fixed.

> As it is now, bogofilter _is_ broken since bad input can cause a crash.

So, bad inputs makes the software abort. The alternative, implemented
now, is to not report the error and forward mail with bogus
classification. That cannot be right.

> The current code is simple and incomplete.  Feel free to suggest
> additions :-)

We need to exit with a failure code in case of error, regardless of the
nature of the error. The simple way to achieve this is letting the
sigjmp handler set a global error variable (global is important here,
because sigjmp is allowed to clobber certain local variables and I've
seen it do this in leafnode) and read that to exit with code 3.

> > ...without any macroscopic error return. What I'd like bogofilter to
> > do is: a. do not change any data bases unless the read + parse was
> > entirely successful b. report the error back and continue to exit with
> > an error exit code.
> 
> Remembering there was an error and not changing databases is fine, and
> easy to add.  Reporting an error is fine.  Exiting with an error code is
> problematical.

It's necessary. I wouldn't want to run a system that doesn't tell me
about errors ultimately. In a procmail or maildrop setup, I don't see
anything that bogofilter might print to screen unless it also exited
non-zero.

If I cared about unreliable systems, I'd run Windows.

> > Am I missing something obvious about the code? Else we've just broken
> > bogofilter.
> 
> Other than the corrected inequality, bogofilter is _less_ broken than
> previously.

It conceals errors with unpredictable results.

Look, if a spammer wants to bypass bogofilter, he'll try to jam the
lexer. We won't notice, the user won't notice because the mail is still
delivered, and we'll get "bogofilter false negative" reports.

I admit that aborting bogofilter is a very hard way to deal with an
internal error, but it's the best approach: we'll get the offending mail
quickly, we can fix the error quickly, and the problem is gone.

If we conceal the error, the user may think everything is fine, we may
not ever know there is a mail that breaks the lexer.

-- 
Matthias Andree

Encrypt your mail: my GnuPG key ID is 0x052E7D95