lexer changes

[Boris 'pi' Piwinger]

David Relson wrote:

[Removing escapes]
>> That's why I did it. I couldn't read the code with all those
>> escaping for totally unclear (actually no) reasons.
> 
> The danger in removing backslashes is that where they're needed is not
> consistent.  Certain characters are special in some places, but not in
> others.  For example "?" is special outside of square brackets, i.e. [],
> but is not special inside them.  The minus sign, "-", is special inside
> of square brackets and needs a backslash before it (unless it's the last
> character, i.e. immediately before the right square bracket).  A left
> square bracket inside of square brackets doesn't need to be escaped,
> while a right square bracket _does_ need to be escaped.

Exactly. It is even worse, since different dialects of
regular expression have different rules. In some you need +
as a quantifier, in some \+, same for ? or \?. So if I see a
\ I alway need to verify if this is doing something here and
if so what. So any superfluous \ let's me believe it is
needed, so I don't understand what's going on. There were so
many expressions I did not understand (there are stille
many), so that I had to work a long time to read them. In
some cases I found that they would not make sense (would
never match), in some cases I found that they were just
irritating, in some cases they were required. Obfuscation.

pi