lexer change

[Piotr KUCHARSKI]

On Tue, Nov 11, 2003 at 09:24:15AM -0500, Tom Anderson wrote:
> When the rules exclude otherwise perfectly good tokens, they are
> essentially acting as a pre-filter.  Excluding 10k binary attachments
> would be an example of a good rule because it has a major impact on
> program/database efficiency.  Excluding a token with a dollar sign is
> not, since Bayes can do the filtering on that just fine.

... also increasing database size and impacting its efficiency.

But in general I agree with you.

p.

-- 
Beware of he who would deny you access to information, for in his
heart he dreams himself your master.   -- Commissioner Pravin Lal
http://nerdquiz.sgh.waw.pl/  -- polska wersja quizu dla nerdów ;)