obscured URL not being tokenized

[David Relson]

On Sun, 21 Dec 2003 11:58:51 -0800
Dan Singletary <dvsing at sonicspike.net> wrote:

> Has any thought been put into not only registering single tokens as 
> bogofilter does now, but registering dual tokens so that "color,
> white" was a token, or "display, none" was a token-- this might
> enhance bogofilters accuracy because often you get enhanced meaning
> from looking at two adjacent tokens .. "click here" comes to mind.
> 
> -Dan

Dan,

Token pairs is actually fairly easy to do.  However the plan is for
bogofilter-1.0 to be the best possible single token filter.  Then we'll
add pairing.  FWIW, Michael O'Reilly is already doing pairs.

David