bogoupgrade.sh [was: problems upon upgrade]

[David Relson]

On Wed, 28 Jul 2004 17:56:44 -0800
jerry wrote:

> 
> > Let's find out what _does_ work.  Let's find out if there's anything
> > in wordlist.db:
> >
> > "bogoutil -d ~/.bogofilter | tee wordlist.txt | wc -l" will dump to
> > file and report line count (token count).
> 
> pwd /home/jerry
>  bogoutil -d ~/.bogofilter | tee wordlist.txt | wc -l
> Can't open file '/home/jerry/.bogofilter'

Sorry.  I should have specified the name of the database rather than the
directory.  Use:

bogoutil -d ~/.bogofilter/wordlist.db | tee wordlist.txt | wc -l

> pwd /home/jerry/.bogofilter
> bogoutil -d .bogofilter | tee wordlist.txt | wc -l
> Can't open file '.bogofilter'
> 
> >
> > "grep relson wordlist.txt" should give counts for my userid.
> >
> > "bogoutil -p ~/.bogofilter relson" should give counts and
> > probability(spamicity) of token "relson".
> 
> bogofilter is not active on these emails due to the fact that ALL
> emails were deleted as spam, there were some 100 emails this morning
> that were deleted,

How were you checking for spam/ham?

> some I am sure were spam, I generally recieve about 50 ligitimate
> emails from lists I have subscribed to, and ALL were deleted. 
> bogofilter, procmail, fetchmail have been disabled until I can begin
> to solve this problem.
> At the present time I am using kmail, with no filtering, and feeling
> the effects of to much spam
> >
> > "echo this is a test of bogofilter and wordlist.db | bogofilter -vvv
> > -H" should give scores for all tokens from the echo command.
> 
>  echo this is a test of bogofilter and wordlist.db | bogofilter -vvv
>  -H 
> Error - bad parameter 'strict_check=no'
> Error - bad parameter 'thresh_index = 0'
> Error - bad parameter 'thresh_stats = 0.0'
> Error - bad parameter 'thresh_rtable = 0.0'
> Error - bad parameter 'algorithm=fisher'

To make the error messages go away, remove (or comment out) the lines in
bogofilter.cf.  Alternatively, use bogofilter's "-C" option to run
without config file.

> X-Bogosity: Ham, tests=bogofilter, spamicity=8.38e-04, version=0.92.0
>                                       n    pgood     pbad      fw    
>                                       U
> "wordlist.db"                       318  0.007144  0.000026  0.003680
> +"bogofilter"                       4400  0.096322  0.003324  0.033359
> +"test"                             2747  0.055666  0.007308  0.116045
> +"this"                            43124  0.719936  0.294922  0.290604
> -"and"                             57143  0.845150  0.518190  0.380089
> - N_P_Q_S_s_x_md                        3  0.999384  0.001060 
> 0.000838
>                                          0.001000  0.415000  0.350000

This looks fine.  Five tokens with various scores were found in the
wordlist.  The FAQ describes the columns of the "-vvv" output.

> Ok, what's next?

Take some messages, run them through bogofilter, and look at the scores,
see if they're reasonable, i.e.

bogofilter -v -d ~/.bogofilter < ham.1
bogofilter -v -d ~/.bogofilter < ham.2
....
bogofilter -v -d ~/.bogofilter < spam.1
bogofilter -v -d ~/.bogofilter < spam.2
....