tilde (was: [cvs] bogofilter/src lexer_v3.l,1.125,1.126)
relson at osagesoftware.com
Thu Dec 11 08:30:25 EST 2003
On Thu, 11 Dec 2003 14:11:52 +0100
Boris 'pi' Piwinger <3.14 at logic.univie.ac.at> wrote:
> David Relson wrote:
> >> >> Exclude tilde from tokens.
> >> Everywhere? Or just TOKENBACK?
> > I changed it in TOKENMID and TOKENBACK. What did you want?
> Looking at the history, tilde, apostrophe and backtick were
> forgotten when expanding [:punct:] to allow exclamation
> points in TOKENBACK:
Likely so. The expansion took several iterations to get it right (or so
> Tilde was allowed in TOKENMID already in revision 1.1.
The revision number is somewhat misleading. When the source files were
moved from bogofilter/ to bogofilter/src, all their revision numbers
became 1.1. Before that, there was a lexer.l file which underwent major
changes and was renamed to lexer_v3.l. So there were many earlier
> I believe the reason are TOKENs like F~R~E~E. That makes
> sense, but I don't have a strong opinion here.
Unfortunately there are also "F*R*E*E", "F R E E", etc. Given how few
tokens contain tildes, I doubt it matters whether they're included or
> I meant only TOKENBACK where it is a mistake, but we
> certainly can think about TOKENMID.
More information about the Bogofilter-dev