compactifying the database

[Matthias Andree]

Anatoly Vorobey <avorobey at pobox.com> writes:

> Hello,
>
> I installed bogofilter-sqlite (a Ubuntu package). It works great,
> and before I ask an actual question, please let me shout a heartfelt
> THANK YOU to the developers. I used to run SpamAssassin, which needed
> heavy tweaking and frequent upgrades to work tolerably; and later
> Thunderbird's own filtering, which was terrible. Bogofilter solves my
> 300-spams-a-day problem.

Oh, while Thunderbird's own filtering (the trainable one) in 1.0.X isn't
exactly on par with bogofilter, it's quite decent for me.  (Consider
removing the training database from the menu.)

> It does seem to slow down a bit after a week of running. wordlist.db
>is 17Mb now and I thought it might need to be compacted somehow.
>However, the bf_compact-sqlite script included in the package doesn't
>wor, as it tries to call * rather than *-sqlite programs; and when I
>changed it to call the right executables, bogofilter complains that it
>doesn't understand --db-transaction.

Well, the script shouldn't be passing that option. Is "TXN" set in the
environment? If so, unset it. Else, check what's in your ~/.bogofilter
directory. The -sqlite package should not be dropping log.* files which
cause bf_compact to add --db-transaction.

If it's growing too fast, remove the "-u" option from bogofilter calls.

Oh, and ubuntu shouldn't be renaming the executables (or should patch
the scripts) if they intend to keep the scripts working. If that's
really the problem, please file a bug report with Ubuntu.

> My question: is there a need/an ability to compact the database in the
>SQLite version, and if so, how?

Well, you won't gain too much. Except if you removed tokens "big time"
with the maintenance options -- after that, an SQL "VACUUM" command
probably would not hurt, which would just be

sqlite3 ~/.bogofilter/wordlist.db 'VACUUM;'

-- 
Matthias Andree