Robinson-Fisher (aka chi) beats Bayes chain rule

[Scott Lenser]

> > A few things to point out, some of which you've already noticed:
> > 
> > 1) You're right, it's extremely dogmatic as the number of features
> > goes up.  You need only a few hundred features to bang into the
> > 10^-308 or so hard-limit of IEEE floating point.
> 
> Yup.  My problem is, of course, that a double between 1 and 1 - 1.0e-16
> is indistinguishable from 1; that's why I get at least 71 1.0's at the
> top of the nonspam list.
> 

A common technique used by AI researchers to get around this is to use the
log of the probability.  So instead of multiplying together a bunch of
probabilities you instead add the log of the probabilities.  Makes it much
more stable numerically and since log is monotonic you can directly compare
the log probabilities to decide which class is more likely.

- Scott