[bogofilter] Improved Calculations

[michael at optusnet.com.au]

Tom Allison <tallison at tacocat.net> writes:
[..]
> Keeping all variables fixed, I modified only one and found the best value.
> Using that best, I then varied another variable and found the best for
> that one.
> And so on.

Note that this is not guarenteed to converge!

> I started with robx=(1.0, 0.1, 0.01) and chose 1.0
> then did block_on_subnets and replace_nonascii_chars and picked yes
> for both.
> Finally I picked robx (0.4, 0.5 0.6) and picked on 0.6.
> 
> This gave the same results as if I went through and selected the best
> set of tests results for all three sample sets across all 24 parameter
> combinations.  I ended up at the same place in much less time.

Well, that's empirical evidence that the 5-surface is strictly
of order 2 or less. :)
 
> Statistically this could be highly illegal, unless these variables are
> independent of each other...

It doesn't actually matter if they're independant or not per se.

This is a reasonably standard optimization problem.

You have  error = f(x,s,md,se,he) where all variables are scalar,
and you want to minimize error.

Note that the commonly used error function isn't a good one
to use (i.e. counting the number of false positives) as
the resulting function won't be either continuous or differentiable.

You're better off using a sum of squares error function.

Then you can use any optimization method you like.
i.e. simulated annealing or walking via the
partial derivatives etc etc.

Something that I've previously played with that relies on
the assumption that no partial derivate is trinomial
or worse is:

        pick a point P.
        pick an epsilon.
        calculate error at P
        calculate all the approx partials
                error at P + epsilon * s-vector
                error at P - epsilon * s-vector
                error at P + epsilon * x-vector
                etc etc.
        Pick the highest partial derivative (say robx)
        P += epsilon * x-vector
        reduce epsilon a bit
        iterate.

Which is a nice standard "walk the slope" method that 
give the above assumption is true will definately converge.

Michael.