memory pre-allocation

Gyepi SAM gyepi at praxis-sw.com
Fri Oct 4 19:50:46 CEST 2002


I now have access to an hppa machine, thanks to Hans Fugal,
and in testing, have discovered that bogofilter is extremely slow
on the architecture: I usually kill the process because I get tired of waiting.

I have traced the problem to the memory pre-allocation that occurs in wordhash.c
When I change the values of N_CHUNK and S_CHUNK to 1 and 0, respectively, which
eliminates any pre-buffering, the program responds much more reasonably, even despite
the more frequent xmalloc calls. I am sure it would be even faster if
I trimmed all the code associated with the pre-buffering.

On the other hand, pre-buffering makes a big difference on x86. The standalone wordhash
program, which is a good exerciser for the code, runs almost twice as fast with 
pre-buffering enabled.

So I am thinking that pre-buffering should be disabled on some non x86 platforms
and enabled on x86. I only have access to hppa, powerpc and x86 so I would appreciate
further testing by people with access to other architectures. 

Note that my tests were not extensive, and affected by available memory, but the results
seem pretty clear nonetheless.

-Gyepi

-- 
Where all men think alike, no one thinks very much.
	--Walter Lippmann  



More information about the bogofilter-dev mailing list