histogram, a.k.a. verbosity level

[David Relson]

Greetings,

As many of  you already know, I've got reasonable printout levels for the 
Graham algorithm for verbosity=1,2, etc.  I've been working on comparable 
functionality for the Robinson algorithm.

The attached file shows how bogofilter responds to the following sets of 
options:

	-g -v
	-r -v

	-g -vv
	-r -vv

	-r -vvv

The first two just print out the "X-Bogosity" line.

"-g -vv" prints out the Graham extrema data (15 most extreme tokens with 
probability, cumulative spamicity, and token).

"-r -vv" prints out a Robinson histogram (showing interval, token count for 
interval, avg probability for interval, cumulative spamicity, and hash mark 
histogram).

"-r -vvv" prints out the Rtable.  Alternatively, "-R bogo.tbl" could be 
used (for a similar effect).

The outputs are all to stdout so that they can be included in the delivered 
email.   My personal copy of bogofilter runs with verbose=2, i.e. "-vv", 
and some extra threshold values so messages with no spam indicators only 
show the "X-Bogosity" line.

I'll probably be adding the code to cvs in a day or two.

David
-------------- next part --------------
 
**** bogofilter -g -v  msg.test 
 
X-Bogosity: No, tests=bogofilter, spamicity=0.002278, version=0.7.6-1027.1028 
 
**** bogofilter -r -v  msg.test 
 
X-Bogosity: No, tests=bogofilter, spamicity=0.336295, version=0.7.6-1027.1028 
 
 pri**** bogofilter -g -vv  msg.test 
 
X-Bogosity: No, tests=bogofilter, spamicity=0.002278, version=0.7.6-1027.1028 
 
#  0.400000  0.400000  63.121.31.110 
#  0.400000  0.307692  delivered-to 
#  0.400000  0.228571  flipdog.com 
#  0.400000  0.164948  from 
#  0.400000  0.116364  fs1.i.flipdog.com 
#  0.400000  0.080706  ftp.flipdog.com 
#  0.400000  0.055292  jobhunter 
#  0.400000  0.037553  oct 
#  0.400000  0.025353  osagesoftware.com 
#  0.400000  0.017046  postfix 
#  0.400000  0.011429  received 
#  0.400000  0.007648  relson 
#  0.400000  0.005112  return-path 
#  0.400000  0.003414  sun 
#  0.400000  0.002278  with 
 
**** bogofilter -r -vv  msg.test 
 
X-Bogosity: No, tests=bogofilter, spamicity=0.336295, version=0.7.6-1027.1028 
 
#       int  cnt      prob spamicity  histogram 
#      0.00   40  0.041188  0.011873  ############################ 
#      0.10   35  0.144202  0.039459  ######################## 
#      0.20   42  0.250499  0.088020  ############################# 
#      0.30   39  0.347486  0.141013  ########################### 
#      0.40   74  0.418468  0.231323  ################################################## 
#      0.50   22  0.559556  0.262952  ############### 
#      0.60   11  0.648215  0.281128  ######## 
#      0.70   13  0.761747  0.307962  ######### 
#      0.80   10  0.865302  0.333357  ####### 
#      0.90    1  0.919971  0.336295  # 
 
**** bogofilter -r -vvv  msg.test 
 
X-Bogosity: No, tests=bogofilter, spamicity=0.336295, version=0.7.6-1027.1028 
 
#            Token              pgood      pbad        fw  invfwlog     fwlog 
#   0  detroit                 171.08         0  0.000001  -0.00000  -13.65886 
#   1  supporting               30.86         0  0.000007  -0.00001  -11.91842 
#   2  gathering                10.16         0  0.000020  -0.00002  -10.81988 
#   3  modular                   8.59         0  0.000025  -0.00002  -10.59676 
#   4  hills                     7.94         0  0.000029  -0.00003  -10.46325 
#   5  documenting               5.34         0  0.000040  -0.00004  -10.12683 

...[snip]...

# 284  ffffff                  280.71      2152  0.884868  -2.16168  -0.12232 
# 285  helvetica               227.98      2017  0.898841  -2.29106  -0.10665 
# 286  efefef                    2.34        23  0.919971  -2.52537  -0.08341 
# 287  P_Q_S_invsum_logsum    0.40138   0.79216  0.336295  -147.270  -450.877 
 
-------------- next part --------------
--------------------------------------------------------
David Relson                   Osage Software Systems, Inc.
relson at osagesoftware.com       Ann Arbor, MI 48103
www.osagesoftware.com          tel:  734.821.8800