[PATCH] BDB cache size [was: ext3fs slowness ...]

[David Relson]

Greg,

Running with a 25MB cache makes a huge difference for me.  Sorted data 
takes 23.82s and shuffled data takes 30.33s.  My recollection is that, 
without the patch, the last shuffle test took 17 minutes.

Your patch works fine for DB3.  I've changed the patch so that CACHEMB is 
number of megabytes, i.e. 25, and the call to db->set_cachesize multiplies 
it by 1024*24.  I'm thinking of a config file option, like db_cachesize=25.

########## Test Results ##########

[relson at osage src]$ test.0204.sh
-rw-r--r--    1 relson   relson    7225924 Feb  4 08:13 spamlist.shf
-rw-r--r--    1 relson   relson    7225924 Jan 31 11:32 spamlist.std

spamlist.std
18.91user 0.45system 0:23.82elapsed 81%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (216major+4171minor)pagefaults 0swaps

spamlist.shf
25.87user 0.62system 0:30.33elapsed 87%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (216major+5905minor)pagefaults 0swaps
-rw-r--r--    1 relson   relson   23617536 Feb  4 08:15 spamlist.shf.db
-rw-r--r--    1 relson   relson   16637952 Feb  4 08:15 spamlist.std.db

########## Patch ##########

Index: datastore_db.c
===================================================================
RCS file: /cvsroot/bogofilter/bogofilter/src/datastore_db.c,v
retrieving revision 1.1
diff -u -r1.1 datastore_db.c
--- datastore_db.c	3 Feb 2003 16:55:14 -0000	1.1
+++ datastore_db.c	4 Feb 2003 13:18:21 -0000
@@ -58,6 +58,9 @@
  #define	DB_OPEN(db, file, database, dbtype, flags, mode) db->open(db, 
file, database, dbtype, flags, mode)
  #endif

+#define	CACHEMB 25
+#define	DB_CACHE(db, mb, bytes, chunks) db->set_cachesize(db, 
mb*1024*1024, bytes, chunks)
+
  /* Function prototypes */

  static int db_get_dbvalue(void *vhandle, const char *word, dbv_t *val);
@@ -124,6 +127,12 @@
  	print_error(__FILE__, __LINE__, "(db) create, err: %d, %s",
  		ret, db_strerror(ret));
  	goto open_err;
+    }
+
+    if ((ret = DB_CACHE(handle->dbp, 0, CACHEMB, 1)) != 0) {
+        print_error(__FILE__, __LINE__, "(db) setcache( %s ), err: %d, %s",
+            db_file, ret, db_strerror(ret));
+        goto open_err;
      }

      /* open data base */