BerkeleyDB

[Adrian Otto]

Gyepi,

> > When I asked ESR about it he replied:
> >
> > | The reason I now use DBM for wordlists is in order to make it possible
> > | to support wordlist persistence in memory with mmap, rather
> > | than through the autodaemon approach.
>
> I made the change against the 0.4 version and offered the patch to ESR.
> The rationale was, as David pointed out, to reduce the load time.
> Judy does not have persistent storage, so the lists had to be
> loaded at startup. This would sometimes take up to 3-4+ seconds in my
> ( ~ 1 million word lists).
> ESR initially tried the autodaemon approach, which
> incurs that cost once, but decided the it was a bad idea and
> instead accepted my DB3 patch.
>
> Given the size of the project and the numbers of people involved,
> I think it may be a good idea to evaluate better approaches to all
> of the current implementations before things jell and it becomes
> more difficult to do so.  I choose DB3 and like it, but would
> certainly consider alternatives. Ideally, alternatives
> should have the same benefits of availability, portability, etc.
> The autodaemon concept may even be worth re-evaluating.

I think that this design change in favor of BerkeleyDB was wise. I think ESR
was right-on with his rationale. BerkeleyDB is mature, and widely portable
already. I think it is a good idea to stay with this and whittle down our
TODO list.

Once we have all the functionality in the system that we want, we should
revisit this issue, and use some of the very good suggestions that have been
discussed on this list. As long as we properly abstract the use of the
BerkeleyDB code so that it can be easily replaced in the future, it should
be safe to leave it in place.

I do advocate a limited scope effort to make abstractions for get_word_value
and set_word_value types of operations. This is already present to some
degree, but could be abstracted a little bit more to make future changes
easier.

Thanks,

Adrian