' or ` at TOKENBACK

[Boris 'pi' Piwinger]

David Relson wrote:

>> > TOKENFRONT	[^[:blank:][:cntrl:]<>;&%@|/\\{}^"*,[\]?=():#+$._'`~![:digit:]-]
>> > TOKENBACK	[^[:blank:][:cntrl:]<>;&%@|/\\{}^"*,[\]?=():#+$._'`~-]
>> > TOKENMID	[^[:blank:][:cntrl:]<>;&%@|/\\{}^"*,[\]?=():#+]+
>> > BOGOLEX_TOKEN	[^[:blank:][:cntrl:]<>;&%@|/\\{}^"*,[\]]+
>> 
>> Please double-check since I backported this from my new
>> version of the lexer.
>> 
>> This breaks some tests t.lexer.mbx and t.maint, t.systest
>> was broken for me already before.
> 
> I've made the suggested changes to my copy of lexer_v3.l.  This evening
> I'll have time to run "make check" and look at what's changed.  If the
> results seem reasonable, I'll commit the lexer change and update the
> test results.

I am not sure about BOGOLEX_TOKEN, which additional
characters can show up there which may not show up in a
TOKEN? Right now we have: ?=():#+

BTW: After so many lexer changes, when will we have the next
release?

pi