bogofilter 1.2.2 crashes with "flex scanner push-back overflow"

[Seth David Schoen]

Hi,

(I sent this message yesterday but it didn't make it to the list,
maybe because the original version had an attachment.)

I'm using Ubuntu Maverick's bogofilter 1.2.2 package to filter spam.
Today I found that bogofilter was crashing with the error "flex scanner
push-back overflow" and not filtering spam.  I identified the two
particular spam messages that were causing this problem, and I found
that they would make bogofilter crash every time.  I've also confirmed
that they make bogofilter 1.2.2 crash on another machine, running FreeBSD,
with no wordlist.db, so I think there is a real bug here.

The spam messages seem to consider of a huge number of (long) separately
koi8-encoded tokens.  Their contents were identical except for the date
and recipient address.  I've posted one of the original messages at

http://www.loyalty.org/~schoen/spam.bz2

-- 
Seth Schoen
Senior Staff Technologist                         schoen at eff.org
Electronic Frontier Foundation                    https://www.eff.org/
454 Shotwell Street, San Francisco, CA  94110     +1 415 436 9333 x107