Transactional Code Completed! 0.92.6+txn2.0 released

[Matthias Andree]

On Wed, 01 Sep 2004, Tom Anderson wrote:

> Doesn't the assumption that "your hardware works well" contradict the
> purpose of the transactional code?

I don't think it does. If your hard disk drive suffers a catastrophic
failure, you'll still need to recover from NAS or tape backups.

If your drive cable is broken and corrupts every 16th bit and CRC isn't
used, data will be lost.

The code will however protect against premature application termination,
segfault through application bugs, rampant OOM process killer in the
kernel, signals delivered to the wrong process, power blackout (as long
as the write cache is off), full disks (currently, these lead to data
base corruption as soon as a page needs to be split) and other of the
more common failures.  If the non-transactional bogofilter crashed in
the middle of a registration, the data base would be corrupt or useless.
You can now recover the state before registration and re-do.

> My system, for one, is very stable, so I
> would only expect the transactional code to be useful if, say, my RAM or
> harddrive were to go bad.  Or is the assumption only in respect to lying
> about I/O completion?  Is there a way to detect this with an install test?

It is about I/O completion and atomicity for the main part. The data
must be on permanent storage when the OS reports "complete", i. e. write
caches will, given the state of current operating systems, usually
interfere.

BerkeleyDB 4.1 and newer (I'm using 4.2, which I'd recommend) can
optionally "checksum" the pages in the data base by means of an SHA-1
digest.


We'll still have to find out if the failure modes of the transactional
code are to our user's taste, there are many places where bogofilter
will just abort in case of conflict and leave the retry up to the user,
it'll currently retry only the most common and most abort-prone
operations.

For scoring mail in a recommended setup, everything's fine, maildrop
will exit with EX_TEMPFAIL and the MTA (Postfix, Courier, Exim,
Sendmail, Qmail) will retry a while later, so in the system as a whole,
it'll work rather well. I expect rough spots in bogoutil and perhaps a
bug or two that escaped my testing; given that we don't have full
coverage through our test suite yet.

-- 
Matthias Andree

Encrypted mail welcome: my GnuPG key ID is 0x052E7D95 (PGP/MIME preferred)