non-representative DB benchmark

[Matthias Andree]

On Sun, 07 Sep 2003, Bob Friesenhahn wrote:

> Many of your benchmark runs are too short to be statistically
> accurate.  Remember that the scheduler runs on a 100HZ clock and this

I don't claim they are accurate or representative. For accuracy, taking
only one sample is fallacious enough to invalidate the result.

> is used to account for user & system times.  Unless the system
> provides dedicated hardware for a high-resolution clock (and this is
> used by the 'time' command) the elapsed time measurements are suspect
> as well.  I don't believe normal PC hardware provides a good
> high-resolution clock.

Should not the TSC in recent pentium-compatible processors provide such
information? I wonder if Linux uses it through wait3(2) or if I need
special user-space software.

> The page-in/page-out values when using the various databases are most
> interesting.  It would be interesting to see how the paging effects
> short runs vs long runs.

Apparently, they are skewed. Running the same command twice on an
otherwise idle machine prints for example:

$ \time wc /etc/hosts
     37     104    1109 /etc/hosts
0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (136major+19minor)pagefaults 0swaps
$ \time wc /etc/hosts
     37     104    1109 /etc/hosts
0.00user 0.00system 0:00.00elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (136major+19minor)pagefaults 0swaps

I wonder if mmap() screws the pagefault figures.

> > * score:
> > 11.00user 5.65system 0:16.87elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
> > 0inputs+0outputs (419major+306minor)pagefaults 0swaps
> 
> Why did BerkeleyDB experience 1215 major pagefaults, whereas QDBM
> experienced only 419 major pagefaults?  With a "warm cache" and
> sufficient RAM, major pagefaults should be close to zero.

Maybe cache is accounted for although it doesn't require I/O, or
/dev/null I/O is taken into account for mapping anonymous memory or
whatever. I don't know.

-- 
Matthias Andree

Encrypt your mail: my GnuPG key ID is 0x052E7D95