0.93.1 and DB_CONFIG flags

[David Relson]

On Wed, 10 Nov 2004 17:09:55 +0100
Matthias Andree wrote:

> David Relson <relson at osagesoftware.com> writes:
> 
> > Matthias,
> >
> > I can probably do a 0.93.1 release this evening.  I'm planning on
> > adding 4 BerkeleyDB related options to bogofilter:
> >
> >     db_lk_max_locks=int
> >     db_lk_max_objects=int
> >     db_log_autoremove=boolean
> >     db_txn_durable=boolean
> >
> > I've roughed in what's needed for the above options.  Affected files
> > are bogoconfig.c, configfile.h, datastore_db.c, and datastore_db.h.
> 
> Can we please only add the db_lk* stuff for 0.93.1?

Can do.  I had already planned on #ifdefs using DB_LOG_AUTOREMOVE and
DB_TXN_NOT_DURABLE to ease some of the problems with different versions
of BerkeleyDB.  After adding the needed code for bogofilter, I realized
that bogoutil doesn't understand long options, e.g.
--db_lk_max_locks=123, and doesn't know anything about bogofilter's
config file.  Guess it's time to cut/paste/revise bogoutil's option
handling.

> db_log_autoremove will only work with BerkeleyDB 4.2 and 4.3, but not
> 3.0 to 4.1 (needs emulation).
> 
> db_txn_durable is only a small fragment of the whole puzzle, and
> already deprecated, BerkeleyDB 4.3 (released earlier today) suggests
> to use the more thorough DB_LOG_INMEMORY instead.
> 
> There are other, less intrusive, options to address the write speed
> issue; the most prominent is "don't use the -u option", the BerkeleyDB
> options are DB_TXN_NOSYNC, DB_TXN_WRITE_NOSYNC and perhaps
> DB_LOG_DSYNC, DB_DIRECT_DB, DB_DIRECT_LOG for machines with fast I/O
> and slower memory copies, older SPARCS for instance.

My bet is that the write speed issue Greg reported is for "bogoutil -l".
I _know_ he doesn't use '-u'.

> > In 0.92.8, calls ds_close() included a "bool nosync" parameter.  Is
> > that needed to support db_txn_durable=no ???
> 
> It doesn't belong there, and we MUST NOT use DB_NOSYNC for data bases
> running with DB_TXN_NOT_DURABLE flags lest we want our database to be
> trashed.

The proper actions can wait until 0.93.2 (assuming 0.93.1 today, or this
weekend).
 
> While DB_LOG_INMEMORY isn't documented to this extent, I'm not sure if
> it's safe to combine with DB_NOSYNC, so we'll clear this flag for
> DB_LOG_INMEMORY as well.
> 
> In the TXN environment, it is safe to call DB->close() with DB_NOSYNC
> _UNLESS_ DB_TXN_NOT_DURABLE (or perhaps DB_LOG_INMEMORY - not sure,
> we'll include this to be on the safe side) or similar are set, because
> the data base updates can be recovered from the log - as long as it's
> on permanent storage.
> 
> The relevant changes for datastore_db.c (and NEWS) are in CVS.