Accuracy is lacking

Fri Feb 14 14:30:05 CET 2003

At 07:40 AM 2/14/03, Nick Simicich wrote:
>At 12:44 PM 2003-02-13 -0800, Tracy R Reed wrote:
>
>
>>I don't get any false positives but it is missing a lot of spam. I would
>>say 3/4 of the spam makes it into my inbox and only 1/4 gets filtered.
>>Looking at the spamicity measurement in the header they all fall very near
>>to 0.5. With bayespam the values for spam were always very high and
>>non-spam very low so there were very few edge cases. Here everything seems
>>to fall right on the line and most of it ends up in my inbox. I always
>>send the misclassified spam back through bogofilter to correct the
>>database but it does not seem to be gaining me anything.
>
>The latest version of bogofilter seems to have seriously moved the 
>line.  I have changed my spam cutoff to .38, and I am still getting no 
>false positives.  I am considering moving it to .34.  This seems to be a 
>uniform change - it has applied to a lot of my spam.

Nick & Tracy,

The definition of ham and spam varies according to your wordlists and your 
messages and the algorithm used.  My bogofilter.cf specifies 
algorithm=fisher, ham_cutoff=0.10, and spam_cutoff=0.95.  As you know from 
the statistics I published yesterday, in January my mail server received 
1744 spam and 5045 ham.  Of these 6789 messages, 257 were scored between 
0.10 and 0.95 - hence were classified as Unsure.

Below are the true (human) classifications, the bogofilter scores, and the 
subjects of the low scorers (under 0.20) and the highers (above 0.80).  As 
you can see, there are low scoring spam and high scoring ham with the 
lowest spam at 0.103074 and the highest ham at 0.884783.  From these 
results, I could set the spam_cutoff at 0.885 without generating false 
positives.  So, while 0.38 may work for you, it won't work for me.

I guess the variability of my results means that I get mail that's very 
mixed as to content, hence is difficult to score.

It'll be amusing to see what score bogofilter gives to _this_ message 
(given its mix of ham and spam subjects).

David

*** Results Follow ***

S 0.103074 1124 Subject: A D V:Vì‚¨"â˜I–Ú‚Ì‚¨'m‚ç‚¹B 5721WROe1-732O-13
H 0.108057 3076 Subject: Re: Sims Unleashed URL
H 0.109672 2309 Subject: JobHunter results from FlipDog.com
S 0.109889 4036 Subject: VERY IMPORTANT!! i found virus!!!
H 0.110648 2324
H 0.111941 2176 Subject: Complimentary Subscription to CIO Insight (A Ziff 
Davis Publication)
S 0.113853 1125 Subject: Best New Tradeshow Display by Opera Portables, 
Inc. adv
H 0.114413 0961 Subject: Nicole
H 0.121052 0942 Subject: [awc-talk] Our next AWC lunch is this Friday...
H 0.122276 2794 Subject: Fwd: Fw: Dr. Suess (Read this dont delete)
H 0.126714 0520 Subject: [awc-announce] Our next AWC lunch is this Friday...
H 0.136722 0394 Subject: asc.2003.01.h.mbx.0211.1950.out 
asc.2003.01.h.mbx.0211.2346.out ...
H 0.138088 3440 Subject: I have your resume and would like to speak to you
H 0.139666 0638 Subject: Save up to 25% on Hammermill Paper, Plus Leather 
Chair Only $49.98
H 0.142527 4872 Subject: A Greeting From NeoPets!
S 0.142780 2022 Subject: Link Confirmation
H 0.144211 4944 Subject: Re: stein roe & others
H 0.144418 0066 Subject: Fwd: Total Signs of Flirting
H 0.144444 4975 Subject: JobHunter results from FlipDog.com
H 0.156238 3864 Subject: Today's Headlines from NYTimes.com Saturday, 
January 25, 2003
H 0.158740 3966 Subject: Today's Headlines from NYTimes.com Sunday, January 
26, 2003
H 0.164018 0578 Subject: Library Notice
H 0.165106 4295 Subject: January 2003 - eSecurity News from McAfee Security
H 0.166402 0815 Subject: Do You Plan Your Business? Quicken's Small 
Business Newsletter
H 0.166568 4008 Subject: Fwd: Fun
H 0.171399 1972 Subject: HP Business Essentials - Winter 2003
H 0.173324 2627 Subject: Re: AOL or Hotmail?
H 0.180260 0632 Subject: TurboTax Newsletter
H 0.180527 2739 Subject: Today's Headlines from NYTimes.com Sunday, January 
19, 2003
S 0.183237 4016 Subject: Re: Tooth?
H 0.185431 0708 Subject: Save 40% on VirusScan Online plus a bonus gift - 
Start the New Year off right!
H 0.186272 2978 Subject: Tame the Holiday Handheld Invasion!
H 0.198368 2020 Subject: Yahoo! ID Search Results
H 0.198889 2190 Subject: Credit Card Expiration Approaching
S 0.103074 1124 Subject: A D V:Vì‚¨"â˜I–Ú‚Ì‚¨'m‚ç‚¹B 5721WROe1-732O-13
H 0.108057 3076 Subject: Re: Sims Unleashed URL
H 0.109672 2309 Subject: JobHunter results from FlipDog.com
S 0.109889 4036 Subject: VERY IMPORTANT!! i found virus!!!
H 0.110648 2324
H 0.111941 2176 Subject: Complimentary Subscription to CIO Insight (A Ziff 
Davis Publication)
S 0.113853 1125 Subject: Best New Tradeshow Display by Opera Portables, 
Inc. adv
H 0.114413 0961 Subject: Nicole
H 0.121052 0942 Subject: [awc-talk] Our next AWC lunch is this Friday...
H 0.122276 2794 Subject: Fwd: Fw: Dr. Suess (Read this dont delete)
H 0.126714 0520 Subject: [awc-announce] Our next AWC lunch is this Friday...
H 0.136722 0394 Subject: asc.2003.01.h.mbx.0211.1950.out 
asc.2003.01.h.mbx.0211.2346.out ...
H 0.138088 3440 Subject: I have your resume and would like to speak to you
H 0.139666 0638 Subject: Save up to 25% on Hammermill Paper, Plus Leather 
Chair Only $49.98
H 0.142527 4872 Subject: A Greeting From NeoPets!
H 0.142780 2022 Subject: Link Confirmation
H 0.144211 4944 Subject: Re: stein roe & others
H 0.144418 0066 Subject: Fwd: Total Signs of Flirting
H 0.144444 4975 Subject: JobHunter results from FlipDog.com
H 0.156238 3864 Subject: Today's Headlines from NYTimes.com Saturday, 
January 25, 2003
H 0.158740 3966 Subject: Today's Headlines from NYTimes.com Sunday, January 
26, 2003
H 0.164018 0578 Subject: Library Notice
H 0.165106 4295 Subject: January 2003 - eSecurity News from McAfee Security
H 0.166402 0815 Subject: Do You Plan Your Business? Quicken's Small 
Business Newsletter
H 0.166568 4008 Subject: Fwd: Fun
H 0.171399 1972 Subject: HP Business Essentials - Winter 2003
H 0.173324 2627 Subject: Re: AOL or Hotmail?
H 0.180260 0632 Subject: TurboTax Newsletter
H 0.180527 2739 Subject: Today's Headlines from NYTimes.com Sunday, January 
19, 2003
H 0.183237 4016 Subject: Re: Tooth?
H 0.185431 0708 Subject: Save 40% on VirusScan Online plus a bonus gift - 
Start the New Year off right!
H 0.186272 2978 Subject: Tame the Holiday Handheld Invasion!
H 0.198368 2020 Subject: Yahoo! ID Search Results
H 0.198889 2190 Subject: Credit Card Expiration Approaching

S 0.804141 0641 Subject: Get Real Simple and a 4-piece Feng Shui Kit FREE!
H 0.812703 3296 Subject: Nicole
S 0.817694 0366 Subject: New Year. New QuickBooks Business Tools.
S 0.828118 1298 Subject: fwd: regarding your sweepstakes entry
S 0.833598 0363 Subject: Is this Robin Williams?
S 0.833604 1514 Subject: Earn 1,000 miles
H 0.839499 4420 Subject: petz for adoption
H 0.861614 3897 Subject: David, recognize any of these 7 names?
S 0.866400 0874 Subject: Get paid to shop Walmart, Dell, Amazon, J. Crew 
and other top retailers
S 0.866554 5017 Subject: Card
S 0.876401 0509 Subject: Free Snow Scene Screen Saver
S 0.878521 0785 Subject: Government "Secured" Certificates That Pay You 
16%, 18%, up to 50% 1223366866
S 0.883016 0826 Subject: Get paid to shop Walmart, Dell, Amazon, J. Crew 
and other top retailers
S 0.883815 0702 Subject: New to SUPERCOMM? Learn more today!
H 0.884783 2590 Subject: David, recognize any of these 11 names?
S 0.886015 1135 Subject: Join the search for $20,000 worth of buried treasure
S 0.886131 0277 Subject: Free Holiday E-cards!
S 0.887868 0025 Subject: Free Holiday E-cards!
S 0.893983 1366 Subject: Bulk email Proxy Mailer 2124xmIm2-213aNbH4442Z-21
S 0.895859 1515 Subject: World's First hand-powered charger for cell phones
S 0.897900 0092 Subject: not 3, not 4, not 5...8 Great Products in 1!
S 0.898857 1077 Subject: Get paid to shop Walmart, Dell, Amazon, J. Crew 
and other top retailers
S 0.899732 1078 Subject: Last Weeks to Enter: Millions Up for Grabs
S 0.901973 1051 Subject: Big Savings On Your Phone Bill With Vonage
S 0.901973 1707 Subject: Big Savings On Your Phone Bill With Vonage
S 0.908066 0810 Subject: The Ultimate Anti-Aging Product
S 0.910962 0806 Subject: The Ultimate Anti-Aging Product
S 0.914671 0853 Subject: Get top sellers at your price with NO obligation!
S 0.917196 1576 Subject: Are your children safe online? Here's the solution.
S 0.918603 1556 Subject: re: Save At Your Favorite Stores
S 0.920894 1546 Subject: re: Save At Your Favorite Stores
S 0.922208 1288 Subject: Prepare Your Taxes FREE with TaxACT.
S 0.922423 0594 Subject: Prepare Your Taxes FREE with TaxACT.
S 0.923355 1289 Subject: Prepare Your Taxes FREE with TaxACT.
S 0.923564 0593 Subject: Prepare Your Taxes FREE with TaxACT.
S 0.932501 4115 Subject: Online Savings Event -- Open Now!
S 0.936265 0909 Subject: We Can Fix Your Credit..... nigsg
S 0.942723 1642 Subject: Increase Your Cellphone Reception
S 0.943954 1414 Subject: Increase Your Cellphone Reception
S 0.946025 1611 Subject: Survey on cars -You can get $20 e-gift coupon!