Bogofilter-1.1.0 - new current release

[David Relson]

Bogofilter v1.1.0 is now available.

It has new parsing options for setting the minimum token length,
maximum token length, max number of words per token (pairs, triples,
etc), and max length of a multi word token.  These options are:

      --min-token-len=N           min len for single tokens
      --max-token-len=N           max len for single tokens
      --max-multi-token-len=N     max len for multi-word tokens
      --multi-token-count=N       number of tokens per multi-word token

These options provide additional flexibility in parsing an email
message.  When used with a count greater than one, the
multi-token-count option does seem to increase bogofilter's accuracy.
However it also increase the number of tokens processed per message,
will _significantly_ increase the size of your wordlist (database),
and _will_ make bogofilter run slower.

########################################################################

Files are available at http://sourceforge.net/projects/bogofilter for
download.

Here are the md5sums for the release:

b669e73dd00a4bf244b5101dd7369465  bogofilter-1.1.0-1.src.rpm
da771cad1d68f9aaff03c736891b2412  bogofilter-1.1.0.tar.bz2
571483848f27d652f860ec169c925127  bogofilter-1.1.0.tar.gz

0ef0467a75e55a9cf7ad624a6eeb57ff  
        bogofilter-db42-1.1.0-1.i586.rpm
9249482268c9b78095cf36b5f8dc8509  
        bogofilter-db42-debug-1.1.0-1.i586.rpm
de3f8fa6d593797d81c174afb3aec52a
        bogofilter-db42-static-1.1.0-1.i586.rpm
8c3e7482b14a690165710c5399c24a79
        bogofilter-db42-static-debug-1.1.0-1.i586.rpm
98c9a96c3b58fd279cd3aa5e4bf8defb  
        bogofilter-sqlite3-1.1.0-1.i586.rpm
2ad93d25f3ade9579d032a2b07b2bf65
        bogofilter-sqlite3-debug-1.1.0-1.i586.rpm
6983d5171bf540efc7224adcbdd53993
        bogofilter-sqlite3-static-1.1.0-1.i586.rpm
af862fd2a9a1ad97bae000a160b81340
        bogofilter-sqlite3-static-debug-1.1.0-1.i586.rpm

################################################################

The default values are documented in bogofilter.cf, i.e.

#### lexer parameters
#
#	minimum and maximum lengths for single tokens
#
#min-token-len=3			# default
#max-token-len=30			# default
#
#	count and length for multi-word tokens
#	Note: if length not specified, defaults to
#	      multi-token-count * max-token-len (approx)
#
#multi-token-count=1			# default
#max-multi-token-len=0			# default

################################################################

Below are a test message, a command that uses all 4 options, and the
command's output.

### MESSAGE ###

From me at bogofilter.org
To:  you at bogofilter.org

This is a test

### COMMAND ###

bogolexer -p --min-token-len 2 --max-token-len 20 --multi-token-count 2
--max-multi-token-len 40 -I msg.110.txt

### OUTPUT ###

head:From
head:me
head:From*me
head:bogofilter.org
head:me*bogofilter.org
to:you
to:bogofilter.org
to:you*bogofilter.org
This
bogofilter.org*This
is
This*is
test
is*test