markovian matching & dobly noise reduction

[Greg Louis]

On 20040226 (Thu) at 0208:35 -0500, Tom Anderson wrote:
> http://yro.slashdot.org/article.pl?sid=04/02/24/0025219&mode=nested
> 
> These methods sound useful... has anyone looked into integrating them
> into bogofilter?
> 
Haven't looked at Markov but I've been doing some experiments with
Dobly.  Not overly encouraging so far: in the best test yet, I got
about a 15% reduction in error rate from a version of bogofilter that
was (a) hacked to use token pairs, and (b) running a slightly
simplified Dobly.  Considering the humungous overhead penalty that
these changes impose, I'd need to find a way to turn that 15% into
something like 90% before it could be worth putting in mainline, and
even then, major performance (speed) enhancement would be essential.

-- 
| G r e g  L o u i s         | gpg public key: 0x400B1AA86D9E3E64 |
|  http://www.bgl.nu/~glouis |   (on my website or any keyserver) |
|  http://wecanstopspam.org in signatures helps fight junk email. |