Robinson-Fisher (aka chi) beats Bayes chain rule

[Greg Louis]

A correction (sorry) and some further information:

On 20021129 (Fri) at 2039:57 -0500, Greg Louis wrote:
> A detailed writeup of this experiment will appear sometime during the
> coming weekend at http://www.bgl.nu/~glouis/bogofilter/FisherBcr.html
> but for now here's a summary:
> 
<snip>
> 
> We can calculate the means and 95% confidence limits of these values,
> and they come out like this:
> 
>     calc meanfnpc lcl95 ucl95
> 1    chi     1.51  1.45  1.56
> 2 bcr-fw     1.79  1.73  1.85

No they don't; I entered the wrong number of degrees of freedom for the
residuals (4 instead of 2).  The actual confidence limits are these:

    calc meanfnpc lcl95 ucl95
1    chi     1.51  1.38  1.63
2 bcr-fw     1.79  1.66  1.92

A corrected graph, file bcr2.png, is attached.

Also, it turns out that I can force the false-positive count down to 71
and still get a meaningful comparison (remember, chi is capable of
operating with a cutoff high enough that only 42 false positives are
reported).  The result of doing this is interesting: chi's
false-negative count climbs only a little, but that of the chain-rule
method is much more strongly affected:
    calc run fneg  percent
1    chi   0   30 2.118644
2    chi   1   25 1.765537
3    chi   2   27 1.906780
4 bcr-fw   0   91 6.426554
5 bcr-fw   1   84 5.932203
6 bcr-fw   2   96 6.779661

    calc meanfnpc lcl95 ucl95
1    chi     1.93  1.25  2.61
2 bcr-fw     6.38  5.70  7.06

The second graph, bcr2a.png, shows this result.


-- 
| G r e g  L o u i s          | gpg public key:      |
|   http://www.bgl.nu/~glouis |   finger greg at bgl.nu |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bcr2.png
Type: image/png
Size: 2727 bytes
Desc: not available
URL: <http://www.bogofilter.org/pipermail/bogofilter/attachments/20021129/5ee5c93a/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bcr2a.png
Type: image/png
Size: 2770 bytes
Desc: not available
URL: <http://www.bogofilter.org/pipermail/bogofilter/attachments/20021129/5ee5c93a/attachment-0001.png>