BOGOLEX_TOKEN

[David Relson]

Hi pi,

Please disregard previous patch.  It unnecessarily allowed some
characters.  Attached is a revised patch that helps clarify the
difference between TOKENMID, BOGOLEX_TOKEN, and TOKENBACK.  Since
BOGOLEX_TOKEN's characters are a subset of TOKENMID's characters and
TOKENMID's characters are a subset of TOKENBACK's characters, the three
patters could be build upon one another.  However I don't think that
helps make the lexer clearer so I'm not inclined to make the change.

David
-------------- next part --------------
Index: lexer_v3.l
===================================================================
RCS file: /cvsroot/bogofilter/bogofilter/src/lexer_v3.l,v
retrieving revision 1.172
diff -u -r1.172 lexer_v3.l
--- lexer_v3.l	26 Nov 2006 17:56:29 -0000	1.172
+++ lexer_v3.l	26 Nov 2006 19:30:03 -0000
@@ -146,9 +146,9 @@
 MSG_COUNT	^\".MSG_COUNT\"
 
 TOKENFRONT	[^[:blank:][:cntrl:][:digit:][:punct:]]
-TOKENMID	[^[:blank:][:cntrl:]<>;=():&%$#@+|/\\{}^\"?*,\[\]]*
-BOGOLEX_TOKEN	[^[:blank:][:cntrl:]<>;    &%  @ |/\\{}^\" *,\[\]]+
-TOKENBACK	[^[:blank:][:cntrl:]<>;=():&%$#@+|/\\{}^\"?*,\[\]._~\'\`\-]
+TOKENMID	[^[:blank:][:cntrl:]:$*<>;=()&%#@+|/\\{}^\"?,\[\]]*
+BOGOLEX_TOKEN	[^[:blank:][:cntrl:]   <>;=()&%#@+|/\\{}^\"?,\[\]]*
+TOKENBACK	[^[:blank:][:cntrl:]:$*<>;=()&%#@+|/\\{}^\"?,\[\]._~\'\`\-]
 
 TOKEN		{TOKENFRONT}({TOKENMID}{TOKENBACK})?