Experience with 0.15.3 (was: qdbm tuning)
sbellon at sbellon.de
Wed Sep 10 13:37:18 EDT 2003
Matthias Andree wrote:
> Stefan Bellon <sbellon at sbellon.de> writes:
> > 2) Matthias re-worked my QDBM code and replaced the Relic API with
> > the Depot API. First of all, this makes a few includes that are
> > still present in the code superfluous, but second and worse,
> > it's a lot slower as the Relic API does some optimization for
> > you that you have to do by your own if you use the Depot API. I
> > have added those things now. See attached patch for
> > src/datastore_qdbm.c.
> Why is the alignsize set to 16? We store 12-byte records at the
> moment, and 16 are "dangerously" aligned to presumed CPU cache lines
> so we'd better avoid it. 12 nicely skews the array mapping to CPU
> cache lines on anything that has cache lines longer than 32 bit and
> should improve cache efficiency without ever causing adverse effects.
Yes, you're right here. Performance is increased by a few percent and
the database is smaller when using 12 as alignment size.
> I don't think we should stuff that re-organization code into
> db_set_setvalue, or is the dpbnum so cheap we can afford it?
Yes, it's really O(1). Depot keeps the bnum and rnum values cached.
> I'd think we'd better use db_close to reorganize. Objections?
In addition it's fine with me. As a replacement, I'd be against it. As
I mentioned, when adding large amounts of data, you'll soon run into a
DB_MAXFILL > 1 value. This slows down till it only crawls along. As it
doesn't cost much, I don't see why it shouldn't be done.
More information about the Bogofilter-dev