bogofilter compilation problems with "make check"
Nick Simicich
njs at scifi.squawk.com
Wed Jan 29 08:39:23 CET 2003
Having just pushed about 4 more bogofilter cheating spams (all base64,
sprinkled with comments) through bogofilter 0.9.1.2, I decided to try to
upgrade to the latest bogofilter (0.10.1.2).
I downloaded bogofilter 0.10.1.2 from sourceforge, specifically
http://unc.dl.sourceforge.net/sourceforge/bogofilter/bogofilter-0.10.1.2.tar.gz.
[root at scifi bogofilter]# ls -l bogofilter-0.10.1.2.tar.gz
-rw-r--r-- 1 root root 418279 Jan 29 00:25
bogofilter-0.10.1.2.tar.gz
[root at scifi bogofilter]# sum bogofilter-0.10.1.2.tar.gz
03515 409
[root at scifi bogofilter]# md5sum bogofilter-0.10.1.2.tar.gz
5c0c2a07b7fb0092bc11ba4b92cea0c4 bogofilter-0.10.1.2.tar.gz
[root at scifi bogofilter]# cksum bogofilter-0.10.1.2.tar.gz
2397712506 418279 bogofilter-0.10.1.2.tar.gz
[root at scifi bogofilter]#
I see that the length of the file agrees with the described length, but I
can't find a published checksum on sourceforge.
The "make" was clean using my usual
./configure --with-db=/root/bogofilter/db-4.1.24/build_unix/
"make check" fails. (The make checks before this all work). Some isolation:
[root at scifi bogofilter-0.10.1.2]# cd tests/bogofilter/
[root at scifi bogofilter]# make check-TESTS
FAIL: t.lexer.mbx
FAIL: t.robx
PASS: t.split
PASS: t.systest
PASS: t.grftest
===================
2 of 5 tests failed
===================
make: *** [check-TESTS] Error 1
[root at scifi bogofilter]#
It looks like the message count patch was already installed. The killer
message patch was already installed in the downloaded version. Applying
the tilde patch did not help.
It looks like the tests might actually be failing this time, as opposed to
the framework simply not working on my system. For t.lexer.mbx, there seem
to be hard coded values in the file, which is supposed to match (for
example) wc -l of good.2. That count is one off expected, the other count
is wrong as well. For t.robx, there is a significant discrepancy in the
output of the program - it expects a certain value for spamicity for a
token, .ROBX, and the value calculated is different.
I am kind of wondering if (for example) the flex I have is generating
something other than expected. The other possibility, of course, is that
there is bad juju in the expected results because something has changed in
the parsing program but the tests were not updated.
I am wondering exactly what this indicates, and if there is any reason I
should not upgrade. The tests in ../tests work although "t.lock1" takes
lots longer. The tests in ../tests/bogoutil work. These seem to correspond
to all the old tests. It may well be that these new tests, well, test
paths that were always sort of broken because of lexer differences on my
system.
Anyway, I will try to make this as easy as possible for anyone who wants to
work on this to work on it.
The files I was working on when this happened were copied to a different
system (scp -r scifi:/root/bogofilter/bogofilter-0.10.1.2/* .) where I run
a webserver and are available at this URL for anyone who wants to work on
this: http://majordomo.squawk.com/njs/bogofilter/
In case it is more convenient to grab a complete copy of this and look at
it on your own system, I did a
tar -cvzf bogofilter.whoops.tgz * and this file is available as:
http://majordomo.squawk.com/njs/bogofilter/bogofilter.whoops.tgz
Output of the failing tests, with sh -x and -v on the test runs are as follows:
[root at scifi bogofilter]# sh -x ./t.robx -v
+ set -e
+ : .
++ pwd
+ relpath=/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../..
+ NODB=1
+ . ./../t.frame
++ set -e
++ : /var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../..
++ : .
++ [ -z ]
+++ basename ./t.robx
+++ sed s@^t.@@
++ NAME=robx
+++ date +%Y%m%d
++ DATE=20030129
++ TMPDIR=./robx.20030129
++ SUPPRESS_DELETE=NO
++ rm -rf ./robx.20030129
++ mkdir ./robx.20030129
++ test xNO = x
++ echo Results are in ./robx.20030129. Remove it after testing.
Results are in ./robx.20030129. Remove it after testing.
++ BOGOFILTER_DIR=./robx.20030129
++ export BOGOFILTER_DIR
++ [ -z 1 ]
++ true
++ MALLOC_CHECK_=2
++ export MALLOC_CHECK_
++ false
+ : awk
+ verbose=0
+ [ -v = -v ]
+ verbose=1
+ [ -d ./robx.20030129/tests ]
+ mkdir -p ./robx.20030129/tests
+ CONFIG=./robx.20030129/test.cf
+ BOGOFILTER=
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogofilter
-c ./robx.20030129/test.cf -y 0
+ BOGOUTIL=
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogoutil
+ SYSTEST=.
+ cat
+
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogofilter
-c ./robx.20030129/test.cf -y 0 -r -s
+
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogofilter
-c ./robx.20030129/test.cf -y 0 -r -n
+ [ ! -z ]
+
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogoutil
-d ./robx.20030129/spamlist.db
+
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogoutil
-d ./robx.20030129/goodlist.db
+
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogoutil
-vvv -R ./robx.20030129
+ [ ! -z ]
++
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogoutil
-w ./robx.20030129 .ROBX
++ awk /.ROBX/ { print $2; }
+ RESULT=252954
+ WANT=253287
+ [ 1 -ne 0 ]
+ echo want: 253287, have: 252954
want: 253287, have: 252954
+ test 252954 = 253287
[root at scifi bogofilter]#
[root at scifi bogofilter]# sh -x ./t.lexer.mbx -v
+ set -e
+ : .
++ pwd
+ relpath=/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../..
+ NODB=1
+ . ./../t.frame
++ set -e
++ : /var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../..
++ : .
++ [ -z ]
+++ basename ./t.lexer.mbx
+++ sed s@^t.@@
++ NAME=lexer.mbx
+++ date +%Y%m%d
++ DATE=20030129
++ TMPDIR=./lexer.mbx.20030129
++ SUPPRESS_DELETE=NO
++ rm -rf ./lexer.mbx.20030129
++ mkdir ./lexer.mbx.20030129
++ test xNO = x
++ echo Results are in ./lexer.mbx.20030129. Remove it after testing.
Results are in ./lexer.mbx.20030129. Remove it after testing.
++ BOGOFILTER_DIR=./lexer.mbx.20030129
++ export BOGOFILTER_DIR
++ [ -z 1 ]
++ true
++ MALLOC_CHECK_=2
++ export MALLOC_CHECK_
++ false
+ OUT=lexer.mbx.out
+ verbose=0
+ [ -v = -v ]
+ verbose=1
+ BOGOLEXER=
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogolexer
+
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogolexer
-p
+ sort -u
+
/var/spool/news/bogofilter/bogofilter-0.10.1.2/tests/bogofilter/../../bogolexer
-p
+ sort -u
++ cat ./lexer.mbx.20030129/spam.2
++ wc -l
++ cat ./lexer.mbx.20030129/good.2
++ wc -l
+ RESULT= 1145. 3074
++ echo 1145. 3074
++ sed s@ @@g
+ RESULT=1145.3074
+ WANT=1147.3075
+ [ 1 -ne 0 ]
+ echo want: 1147.3075, have: 1145.3074
+ [ ! -z ]
+ test 1145.3074 = 1147.3075
[root at scifi bogofilter]#
--
SPAM: Trademark for spiced, chopped ham manufactured by Hormel.
spam: Unsolicited, Bulk E-mail, where e-mail can be interpreted generally
to mean electronic messages designed to be read by an individual, and it
can include Usenet, SMS, AIM, etc. But if it is not all three of
Unsolicited, Bulk, and E-mail, it simply is not spam. Misusing the term
plays into the hands of the spammers, since it causes confusion, and
spammers thrive on confusion. Spam is not speech, it is an action, like
theft, or vandalism. If you were not confused, would you patronize a spammer?
Nick Simicich - njs at scifi.squawk.com - http://scifi.squawk.com/njs.html
Stop by and light up the world!
More information about the Bogofilter
mailing list