[PATCH] lexer fix for base64 and CRLF

[David Relson]

Allyn,

This form of patch submission is good.  I have committed your lexer.l fix 
into cvs.

David

At 12:19 PM 10/19/02, Allyn Fratkin wrote:
>this is a fix for a problem i reported in bug#618368, where base64 data
>is not recognized and ignored by the lexer if the input lines end in CRLF
>(e.g., if training bogofilter with mailboxes created on Windows).
>currently, the base64 data is treated as normal input data, parsed
>and learned, causing major bloat of the word lists.
>
>this is a quick fix, perhaps not the correct long-term fix, but that would
>require someone a lot more skilled with lex/flex than myself.  there is
>mention in lexer.l comments about ignoring carriage-returns in the input
>but for whatever reason, at the point that base64 data is recognized, the
>carriage return is still present in the input line.
>
>this is a three-character fix that adds \r? to the regular expression that
>checks for a base64 input line.
>
>please let me know if there is any way i could be sending these in a more
>helpful way.  thanks.
>--
>Allyn Fratkin             allyn at fratkin.com
>Escondido, CA             http://www.fratkin.com/