invalid html warfare

[Jef Poskanzer]

>What if we removed from the database each token occurring only once in the 
>database?  (bogoutil -c 1  *.db??)This would only be practical if done on a 
>sufficiently infrequent interval for "good data" to accumulate more than one 
>hit, but often enough to prevent database pollution.

Yeah, I was thinking when I get around to running a weekly bogoutil
cleanup cron job, I'd have it get rid of singletons that are older
than a week.  Probably also nuke any token older then a few months.
(I'm using -u.)  But the databases really aren't big enough to
worry about yet.
---
Jef

         Jef Poskanzer  jef at acme.com  http://www.acme.com/jef/