URGENT: need help with analying robinson breakage on FreeBSD

[Matthias Andree]

Hi,

up to some minutes ago, the robinson code was broken on FreeBSD and gave
bogus results, and I'm confident other operating systems suffered as
well.

The cause is that the compare_rstats_t function was broken and did not
meet qsort's requirements, in that in only returned from {zero; one},
while qsort needs {negative, zero, positive}. glibc's qsort seemed to
cope with that, FreeBSD's did not.

Does this only affect the histogram or does that also break
classification in more complex cases? We have the same spamicity here,
but certainly a bogus histogram.

before fix:
X-Bogosity: No, tests=bogofilter, spamicity=0.527095
  int  cnt    prob   spamicity  histogram
 0.00    0  0.000000  0.000000
 0.10    0  0.000000  0.000000
 0.20    0  0.000000  0.000000
 0.30    0  0.000000  0.000000
 0.40    0  0.000000  0.000000
 0.50    0  0.000000  0.000000
 0.60    0  0.000000  0.000000
 0.70    0  0.000000  0.000000
 0.80    0  0.000000  0.000000
 0.90  283  0.551757  0.527095  ##################################################

reference (met after fix):
X-Bogosity: No, tests=bogofilter, spamicity=0.527095
  int  cnt    prob   spamicity  histogram
 0.00   30  0.001954  0.000376  ################
 0.10    2  0.179087  0.002823  ##
 0.20    2  0.276889  0.006772  ##
 0.30   13  0.367564  0.040115  #######
 0.40   99  0.430007  0.222740  ##################################################
 0.50   26  0.539773  0.263274  ##############
 0.60   38  0.672661  0.330365  ####################
 0.70    4  0.756829  0.337797  ###
 0.80   15  0.841556  0.370644  ########
 0.90   54  0.973740  0.527095  ############################

I have no clues whether this affected fisher as well.

I looked at rstats.c, and found that bogofilter 0.8.0 is also affected.

If this only breaks -v output, we can live with that, but if it impairs
the robinson calculations (I still didn't have time to look at the
algorithm), we might want to release 0.8.1.

-- 
Matthias Andree