lexer change

[David Relson]

On 11 Nov 2003 09:20:08 -0500
Tom Anderson <tanderso at oac-design.com> wrote:

> On Tue, 2003-11-11 at 09:04, Boris 'pi' Piwinger wrote:
> > My test yesterday actually showed it does not help to allow
> > those tokens.
> 
> Yes, but your tests are always going to be limited to current or
> recent emails.  What about future emails?  The main benefit of the
> Bayesian method is that it's not hindered by aging of rules like
> SpamAssassin is.  We shouldn't be deciding based on a few more
> incorrect classifications here or there to institute a new rule.  It
> should be a drastic difference, as in >10%, to even consider it.  Who
> decided on the"[^[:blank:][:cntrl:][:digit:][:punct:]]" rule, and why?
>  I might agree
> with a rule if there were a fundamental underlying philosophical
> reason, but just tweaking the output is not a good enough reason.
> 
> Tom

Eric Raymond wrote that rule.  

Out of curiosity, what would _you_ classify as a 10% improvement?