fasten your seatbelts, the database fix merge is due NOW

[Matthias Andree]

Ok,

the locking-fixed and cleaned up version has been committed to a new
branch, bf-092-database-1, the trunk is unchanged. I decided to commit
to a branch to not cause Gyepi conflicting changes before he's to merge
his stuff. A small accident happened (CVS confused branch and trunk and
killed datastore_db.h from the trunk -- it has now been restored.).

How does the locking work? We open a separate file at db_open time and
lock that, and to NEVER change the locking status of the data base,
because we don't have documentation on how to tell an open DB with
application locking to flush its caches and re-read. It's not necessary
either, we know the modus operandi after reading the command line.

All the lock_reader_list and so on are gone, the locking and unlocking
is now the business of db_open and db_close. BTW, not all parts of the
old code were deadlock-proof, and I haven't fixed one of those bugs
yet. (future work).

I cleaned up things by killing datastore_db.h -- no tool has any
business knowing the physical structure or access methods of our
datastore, we have database.h for that, and I introduced db_foreach that
calls a user-specified function for each data item in the data base (I
didn't go for the iterator approach, too much hassle). This involved
major changes to maint.c and bogoutil.c that shrank a bit -- no more
cursor code replication. In fact, the only cursor stuff is in
datastore_db.c where it belongs. Parts outside datastore_db are unaware
of the internal types and structures and just use the opaque void *
type, so this half of the datastore abstraction has been restored. The
other part would be to pull the field splitting (count/date) out of
datastore_db into a class of its own.

make check passes on SuSE Linux 8.1 x86 gcc 3.2, FreeBSD 4-STABLE x86
2.95, Solaris 8 sparc with gcc 2.95 and SunPro 6, "make distdir" passes
on Solaris 8, "make distcheck" passes on SuSE Linux.

The parts that aren't tested automatically by make check need testing!

Off to bed now...


Gyepi, I invite you to have a look at the branch, I'd recommend you
check my branch out to a separate directory:

 cvs -d :ext:LOGIN at cvs.bogofilter.sourceforge.net:/cvsroot/bogofilter \
 co -d DESTINATION -r bf-092-database-1 bogofilter

Change LOGIN to your sourceforge login and give a nonexistent directory
as DESTINATION.

Here's the diffstat:

# Makefile.am      |   10 -
# bogofilter.c     |    6 
# bogoutil.c       |  323 ++++++++++++++++++++-----------------------
# config.c         |    8 -
# configure.ac     |  231 +++++++++++++++++++++++++++++++
# configure.in     |  230 -------------------------------
# datastore.h      |   65 +-------
# datastore_db.c   |  346 +++++++++++++++++------------------------------
# datastore_db.h   |   24 ---
# doc/bogoutil.xml |    1 
# globals.c        |    8 +
# globals.h        |    3 
# main.c           |    5 
# maint.c          |   88 ++++-------
# mime.c           |    8 -
# register.c       |    6 
# tests/t.lock2    |    3 
# wordlists.c      |   68 +++++----
# wordlists.h      |    6 
# 19 files changed, 621 insertions(+), 818 deletions(-)

-- 
Matthias Andree