ALPHA [was: lexer change]

[Boris 'pi' Piwinger]

David Relson wrote:

> Since the latest CVS changes typically take a while to become available,
> I've attached the latest revision.

Great improvements in there. Here are some more:

0) Allowing two-byte-tokens (see my test on the other list)

1) Some \ slipped back in. Out again.

2) I found some more unneeded \.

3) The comment of 09/01/03 does not fit the context. I
   *don't* change this one.

4) HTMLTOKEN definition is doing nothing.

5) By 0) TOKEN_12 reduces to TOKEN_1. Since there was really
   nothing left to T12 than one class I merge it into
   TOKEN_1.

All changes are tested to the best of my knowledge and
database. Please run a short test anyway.

I cannot find the price range rule which is announced in the
comment of 12 May 2003.

Why "NAME lexer_header.l"?

I am not sure about  MSG_COUNT	^\".MSG_COUNT\" -- are those
\ needed?

It is still inconsistent if we use [:digit:] or [0-9] and
similar things.

pi
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: lexer-cleanup
URL: <https://www.bogofilter.org/pipermail/bogofilter-dev/attachments/20031111/64dfc115/attachment.ksh>