test scripts [was: testing min_dev vs tag_header_lines]

[Greg Louis]

On 20030217 (Mon) at 0956:59 -0500, David Relson wrote:
> At 06:32 AM 2/17/03, Boris 'pi' Piwinger wrote:
> 
> >Hi!
> >
> >Some of us do some testing once in a while (does that count
> >as sports?). It would be nice to have an "official" test
> >script (shell or perl) that would produce nicely readable
> >results. So we don't always have to understand the
> >presentation of the results.
> >
> >pi
> 
> pi,
> 
> I've been thinking along the same lines.  Over the weekend I was working 
> with Greg's scripts (which he made available at 
> http://www.bgl.nu/~glouis/bogofilter/subjtab.html#appA).  Greg prefers to 
> use multiple executables (typically one with the option enabled and the 
> other with the option disabled) while I prefer to use the standard 
> executable with config files to enable/disable options.  His scripts are 
> named mkdb, distrib, and runex and paths and program names correct for his 
> environment.  As a result of this weekend's work, I have versions of his 
> scripts that use the standard executable and config files and that give 
> comparable results.
> 
> Greg,
> 
> Do you think the scripts would be an appropriate basis for a test framework 
> (to give them a grandiose title)?

Well, I've been using (mine) that way for the last four or five
experiments I've done.  But they were cobbled together kind of ad hoc. 
If there were someone with time and enthusiasm to look at the lovely
job the Spambayes folk did with their testing environment and build us
something similar, that might be a much better solution ;)
Experimental design is a complex subject, and I'm no guru at it.

That said, of course anyone is free to use what I've posted in the
various experimental writeups that http://www.bgl.nu/bogofilter links
to, or to write generic versions thereof.  Glad if it helps.  If it
breaks, you get to keep both pieces, as usual.

-- 
| G r e g  L o u i s          | gpg public key:      |
|   http://www.bgl.nu/~glouis |   finger greg at bgl.nu |
| Help free our mailboxes. Include                   |
|        http://wecanstopspam.org in your signature. |