lexer change

[David Relson]

On Thu, 06 Nov 2003 21:18:43 +0300
Evgeny Kotsuba <evgen at shatura.laser.ru> wrote:

> Boris 'pi' Piwinger wrote:
> 
> >>    With this change the generated lexer_v3.c file shrinks from 1.8M
> >to>1.2M and a stripped bogofilter executable shrinks from 1.8M to
> >1.4M.>    
> >>
> >
> >Attached is another patch which reduces my bogofilter
> >(executable) from 1.9M to 1.3M.
> >  
> >
> Hmmm...
> I have now with last changes from CVS  lexer_v3.c 111 472 bytes long
> and exe reduces from about 700K to 267K.
> Visually it is working
> 
> SY,
> EK

The two recent lexer changes that remove repetition counts in the TOKEN
and MIME_BOUNDARY patterns make it considerably smaller.  With the
changes, bogofilter passes "make check" and works properly with all my
messages (approx 100,000), with pi's messages (approx 45,000) and now
Evgeny's messages.   This is all to the good.  However, having it work
for lots of messages is not a guarantee that it will work properly for
_all_ messages and I have a lingering concern that the change will cause
a problem at some future point.  Unfortunately, all we can do is wait
and see :-(

On my linux box, running "strip bogofilter" reduces the file size from
748K to 130K.

David