CRLF bugs back in 0.15.*

[Matthias Andree]

On Thu, 11 Dec 2003, Allyn Fratkin wrote:

> i've been using bogofilter 0.14.5.4 for awhile.  my first try with
> bogofilter 0.15.* (specifically 0.15.10) was a disaster
> as CRLF is no longer recognized as a line separator.  this
> causes the same old behavior from around the 0.7 days where
> mime is not discarded (and subsequently causes extreme swelling
> in the word list) and message separators are not recognized.

The UNIX line terminator is LF, not CRLF; other than that, the current
CVS version is fine here:

$ printf "line:\r\nseparator:\r\n\r\nbody\r\n" | bogolexer
normal mode.
get_token: 1 "head:line"
get_token: 1 "head:separator"
get_token: 1 "body"
3 tokens read.