randomtrain vs bogotrain.pl

[Boris 'pi' Piwinger]

David Relson <relson at osagesoftware.com> wrote:

>> >> >> >                    training            scoring
>> >> >> >                  spam  good       spam        good
>> >> >> > bogofilter       75   585      101 (100%)  616 (100%)
>> >> >> > randomtrain      24    27      101 (100%)  616 (100%)
>> >> >> > bogotrain.pl      7     4       94 ( 93%)  605 ( 98%)
>>
>> >The scoring test had 101 messages originally classified spam and 616
>> >originally classified ham.  None of them were Nigerian.  The hoped for
>> >result was to have all 717 messages classified as non-Nigerian.  The two
>> >larger wordlists got this 100% right,
>>
>>I don't understand: Above it says, 616 were recognized as
>>non-Nigerian (good) and 101 as Nigerian (spam). So there are
>>101 false classifications.
>
>Normally I use bogofilter to separate spam and ham.  Nigerian scams are one 
>group of messages that I classify as spam.  As I wanted to know if 
>bogofilter could be used to detect Nigerian scams, I scanned my spam 
>folders and found 75 Nigerian scam messages.  Then I trained bogofilter 
>with the 75 Nigerian spam and 585 ham (in the non-UCE sense).  After that 
>training, I scored 717 messages.  Of the 717 messages, 101 came from my 
>spam corpus (in the UCE sense) and 616 came from my non-spam corpus.  None 
>of the 717 messages were Nigerian spam.
>
>I was curious to see how well the 3 different wordlists would do in 
>recognizing these 717 messages (none of which were Nigerian spam).  The 
>first two wordlists got perfect scores - all the messages were correctly 
>scored as _not_ Nigerian spam.  

How does bogofilter say that a message is Nigerian spam? By
saying, it is spam, since this is how you did train it. And
your above table does show, that not all 717 were classified
as good, i.e., non Nigerian. Instead 101 were classified as
spam which is a synonym for Nigerian.

pi