lexer changes

[Andras Salamon]

On Tue, Nov 11, 2003 at 02:59:01PM +0100, Boris 'pi' Piwinger wrote:
> -ENCODED_WORD	=\?{CHARSET}\?[bq]\?[^\?]*\?\=
> +ENCODED_WORD	=\?{CHARSET}\?[bq]\?[^?]*\?=

Personally I would prefer some of the "extra" backslashes to stay.
The ^\? may be equivalent to ^? in the above regex, but the second version
looks to me like the representation of ASCII DEL (0x7F).  Confusing.

It's great to have this level of scrutiny of the lexer, but please don't
change stuff just to achieve some kind of "optimality".  Mortals like
me still need to read the code.

Simplifications and clarifications of the state machine, on the other
hand, are always useful.

-- Andras Salamon                   andras at dns.net