Bogofilter simulator.

[Matthias Andree]

Petri Kutvonen <Petri.Kutvonen at cs.Helsinki.FI> writes:

> I don't this is a ext3 (or ext3 + ide) problem: See:
>
> kutvonen at orator:~/sources/simbf$ /usr/bin/time ./simbf >/tmp/boo
> 0.17user 4.27system 0:05.33elapsed 83%CPU (0avgtext+0avgdata 0maxresident)k
> 0inputs+0outputs (78major+8minor)pagefaults 0swaps
>
> This machine is a 2.67 GHz Intel P4 with an Intel D845EBG2 motherboard running 
> Red Hat kernel 2.4.18-17.7.xbigmem.

DMA is not an issue on my system, the ext3fs commit interval however may
be an issue, particularly on slower machines -- we're rewriting the same
block 100 times statistically with simbf, and also with Greg's training
data set, and these many overwrites are committed (somewhat slow) only
to be overwritten later.

My machine effectively throws 90 MB/s at the file system which is WAY
more than the drive can handle (about thrice to four times what it takes
in), so the kernel seems to discard like 2/3 or 3/4 of my
overwrites. All this is still is still much slower compared to sorted
input which causes maybe 1 or 2% overwrites, not 99%.

-- 
Matthias Andree