ALPHA [was: lexer change]

[David Relson]

On Tue, 11 Nov 2003 00:15:37 +0100
Boris 'pi' Piwinger <3.14 at logic.univie.ac.at> wrote:

> David Relson <relson at osagesoftware.com> wrote:
> 
> >A1 is needed for the places where a single letter needs to be
> >identified for use in a token and a2 is needed for a single letter
> >followed by a letter a digit.  An example is a token split by an html
> >comment, i.e."T<!xxx>ha<!xx>t".
> >
> >I have corrected the problems and updated CVS.
> 
> I haven't seen it, but could we use [[:alpha:]][[:alnum:]]?
> for a combined version of those? If so then why not write it
> into TOKEN_12 directly. I think it increases readability if
> things which occur only once are not hidden in another
> definition: TOKEN_12 ({TOKEN}|[[:alpha:]][[:alnum:]]?)
> 
> pi

The combined version seems to work fine.  I've tested it and it passes
"make check".  I'm not going to merge it all into TOKEN_12 since (for
the most part), bogofilter's lexer patterns are either high level
(reference other patterns defined for bogofilter) _or_ low level (using
flex's basic character definitions).

Since the latest CVS changes typically take a while to become available,
I've attached the latest revision.

David
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lexer_v3.l
Type: application/octet-stream
Size: 11911 bytes
Desc: not available
URL: <https://www.bogofilter.org/pipermail/bogofilter-dev/attachments/20031110/07907ba6/attachment.obj>