The "Real Deal" [was: challenge ...]

[David Relson]

Michael,

No more training mission.  Now it's on towards the real deal.  I hope
you're up for part 2!!!

I've applied your lexer changes to bogofilter's full parser.  It works
great for the simple tests (from challenge.0904) and for all of "make
check" except one case.  Test message
src/tests/bogofilter/inputs/split.d/msg.dr.0118.base64 used in
tests/bogofilter/t.split has a problem.  This message is multipart mime
with base64 in the first mime part.  Bogofilter-0.14.5.4 can handle it
just fine, but the new unfolding lexer gets one detail wrong - the first
line of the base64 text isn't decoded.  The problem revolves around the
fact that base64 decoding only happens in body parts and the state flag,
i.e. msg_header, is set to true when the first base64 line is read. 
This in turn causes the line to be read as is and not be decoded.  I
think all it needs is a small change to the processing of the blank line
(that ends the mime part header) and the fact that msg_header doesn't
change state at the right moment.  The challenge is to find the change
(likely a one liner) that fixes the parser.

Attached are two tarballs.  challenge.0905.tgz is an updated version of
yesterday's challenge.0904.tgz and contains additional msg.?.txt files
and their msg.?.out files.  File bogo.0905.tgz contains my current
copies of lexer_v3.l, lexer_c, token.c, and token.h (for placement in
the bogofilter/src directory).  If you unpack challenge.0905.tgz in
(new) directory bogofilter/src/test.d, you should be able to run
"test.sh ../bogolexer" and it'll run all the msg.?.txt tests.

I need to turn in for the night, so I'll be able to work tomorrow.

Good luck!  And thank you for the work you've done already.  I've been
going around in circles with these parser changes - ever near the goal,
but not quite reaching it.

David
-------------- next part --------------
A non-text attachment was scrubbed...
Name: challenge.0905.tgz
Type: application/octet-stream
Size: 3158 bytes
Desc: not available
URL: <https://www.bogofilter.org/pipermail/bogofilter-dev/attachments/20030905/976facf9/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bogo.0905.tgz
Type: application/octet-stream
Size: 9403 bytes
Desc: not available
URL: <https://www.bogofilter.org/pipermail/bogofilter-dev/attachments/20030905/976facf9/attachment-0001.obj>