sqlite problems on Mac OS X 10.4

[Jim Correia]

On Sep 9, 2005, at 12:19 PM, Matthias Andree wrote:

> Jim Correia wrote:
>
>> Does BDB have different performance characteristics or semantics?  
>> If  I
>> build with the BDB backend, t.lock3 passes very quickly.
>
> Yes, concurrency was removed from the Berkeley DB backend, albeit  
> for different
> reasons (namely that the lock table sizing was difficult to get  
> right).

bogofilter ensures data integrity using some other mechanism for  
multiple concurrent processes when using BDB?

> I don't know and know no way to tell if the check is just making  
> slow progress
> or none at all. Finding out the amount of concurrency that triggers  
> bugs,
> finishes reasonably fast on intact systems and doesn't take  
> excessively long on
> older systems is a bit difficult.

I think there is a bug here (perhaps in sqlite and/or how the locking  
mechanism works on Mac OS X) but I don't know how to proceed further.  
If I ratchet the test down to 2 concurrent instances, it still hangs.

Both have identical backtraces (easier to verify with only 2 :-):

#0  0x9001efe8 in select ()
#1  0x000161f4 in bf_sleep (delay=4) at system.c:58
#2  0x0000dfdc in rand_sleep (min=1000, max=1000000) at rand_sleep.c:20
#3  0x000157bc in busyhandler (dummy=0x4, count=0) at  
datastore_sqlite.c:247
#4  0x9436b188 in sqlite3pager_pagecount ()
#5  0x9436b520 in sqlite3pager_ref ()
#6  0x9436cc4c in sqlite3pager_sync ()
#7  0x94357ff8 in sqlite3BtreeSync ()
#8  0x94380f30 in sqlite3VdbeSetColName ()
#9  0x943812f0 in sqlite3VdbeHalt ()
#10 0x9437ca80 in sqlite3VdbeExec ()
#11 0x9437f26c in sqlite3_step ()
#12 0x94385a0c in sqlite3_exec ()
#13 0x00015420 in sqlexec (db=0x4, cmd=0x27b84 "COMMIT;") at  
datastore_sqlite.c:144
#14 0x0000c804 in close_wordlists (commit=4) at wordlists.c:259
#15 0x0000a130 in bogomain (argc=8, argv=0x0) at bogomain.c:80
#16 0x00003324 in main (argc=8, argv=0xbffff6b0) at main.c:30

If I hardcode bf_sleep to use 10 microseconds rather than the passed  
in value (in an attempt to make things complete faster) I get

retrying registration after avoided deadlock...
...[repeat many times]
retrying registration after avoided deadlock...
retry count exceeded, giving up.
#### Part 2 ####

And we end up right back where we started.

I'm willing to poke at the problem some more, but I'm not sure where  
to go next.

Jim