lexer problem

[David Relson]

pi,

Please, when you're changing to a new subject either modify the Subject:
line or start a new thread.  Introducing a bug report in the middle of
the long "Much simplified lexer" thread is a bad practice.  The error
report is likely to be overlooked.

Anyhow, using _my_ copy of bogolexer the output looks good.  Attached
are copies of the test message and the output.  If there's an error in
the output, you need to point it out to me.  Note:  I am not using your
lexer.

David
-------------- next part --------------
>From Kerschma at yahoo.com  Sat Jan 18 16:53:41 2003
Return-Path: <Kerschma at yahoo.com>
Delivered-To: john at example.com
Received: from www.kurashi.ne.jp (kura1.kurashi.ne.jp [216.98.107.249])
	by example.com (Postfix) with ESMTP id 1C3D82838C
	for <john at example.com>; Sat, 18 Jan 2003 16:53:41 -0500 (EST)
Received: from Paulo ([218.24.129.170])
	by www.kurashi.ne.jp (8.8.8+2.7Wbeta7/8.8.7) with SMTP id GAA21516;
	Sun, 19 Jan 2003 06:54:06 +0900
Message-Id: <200301182154.GAA21516 at www.kurashi.ne.jp>
Reply-To: "Paul Mendell" <Kerschma at yahoo.com>
From: "Paul Mendell" <Kerschma at yahoo.com>
To: john at omniplanar.com
Subject: =?ISO-8859-3?B?ZXJpYyxBbWF6aW5nIFBlbmlzIERldmVsb3BtZW50IHByb2dyYW0gIQ==?=
Date: Sat, 18 Jan 2003 22:01:13 GMT
Mime-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_NextPart_0119030601"
X-Mailer: Microsoft Outlook Express 6.00.2462.0000

------=_NextPart_0119030601
Content-Type: text/html; charset="iso-8859-1"
Content-Transfer-Encoding: base64

PGh0bWw+DQoNCjxib2R5IGJnY29sb3I9IiNGRkZGRkYiIHRleHQ9IiMwMDAwMDAiPg0KU0k8IS0t
bi0tPlpFIEFORCBTPCEtLUtoYWxpay0tPlRBTUlOQSBETyBNPCEtLUtob3Nyb3dqLS0+QVRURVI8
YnI+DQpNb3JlIFRoYW4gWW91IENhbiBQb3NzaTwhLS1haC0tPmJseSBJbWE8IS0tS2hvdXJ5LS0+
Z2luZSEhPGJyPg0KU2hlIGlzIGp1c3QgdHJ5PCEtLUtpbmctLT5pbmcgdG8gc3BhcmUgeW91ciBm
ZWU8IS0tS2luZy0tPmxpbmdzIGJ5IHRlbGxpbmcgeW91IG90aGVyPCEtLUtpbnNlci0tPndpc2Uu
PGJyPg0KRDwhLS1Lby0tPk9OJ1QgV0FJVCBVTlQ8IS0tS28tLT5JTCBTSEUgSVMgR09ORTxicj4N
ClRPIEZJTkQgT1VUIFRIQVQgWU9VIENPVUw8IS0tS29sbG1hbi0tPkROJ1QgU0FUSVM8IS0tS29u
ZG8tLT5GWSBIRVIhISENCjxwPklOVFJPRDwhLS1Lb3JzaGFrLS0+VUNJTkcsIFRIRSBGSVJTVCBB
TEwtSU4tT05FPGJyPg0KICBNYTwhLS1Lb3VuZGlueS0tPmxlIFBlcmZvPCEtLWEtLT5ybWFuY2Ug
RW5oYTwhLS1LdW1hci0tPm5jZXIgQU5EIFBlPCEtLUt1bXJhLS0+bmlzIEVubGE8IS0tTGFjaG5l
eS0tPnJnZW1lbnQ8YnI+DQogIFdJVEggVEhFIFRSQUQ8IS0tTGFtLS0+RU1BUktFRCBTV0VEPCEt
LUxhbi0tPklTSCBJTkdSRTwhLS1MYXphcm8tLT5ESUVOVCBDRVJOPCEtLUxlLS0+SVRJTi48L3A+
DQo8cD5TdGFuPCEtLUxlYXZlbmdvLS0+ZGFyZCBSZTwhLS1vZC0tPnN1bHRzIEluY2x1ZGU6PC9w
Pg0KPHA+UEVOPCEtLUxlZS0tPklTIEVOTEFSR0VNRTwhLS1MZWUtLT5OVDxicj4NCiAgPT09PT09
PT09PT09PT09PT09PTxicj4NCiAgR0k8IS0tTGVlLS0+UlRIOiAwLjI1JnF1b3Q7IHVwIHRvIDIm
cXVvdDs8YnI+DQogIExFTkc8IS0tTGVlLS0+VEg6IDEmcXVvdDsgdXAgdG8gMy4yNSZxdW90Ozwv
cD4NCjxwPk1BPCEtLUxlZS0tPkxFIFBFUkZPUjwhLS1MZW5nLS0+TUFOQ0UgRU5IPCEtLUxlb25h
cmQtLT5BTkNFUjxicj4NCiAgPT09PT09PT09PT09PT09PT09PT09PT09PT09PT08YnI+DQogIFNU
PCEtLUxldW5nLS0+QU1JTkE6IFVwIHRvIDc0JSBoYXI8IS0tTGVWaW5lLS0+ZGVyIGVyZWN0PCEt
LUxldy0tPmlvbnMsIHRoaXMgd2lsbCBoZWw8IS0tTGV3LS0+cCBjb250cmlidTwhLS1MZXdpcy0t
PnRlIHRvPGJyPg0KICBsb248IS0tTGV3aXMtLT5nZXIgc2U8IS0tTGV6Y2Fuby0tPnh1YWwgZXhw
ZTwhLS1MaS0tPnJpZW5jZXMuPGJyPg0KICBDTEk8IS0tTGlhbmctLS0+TUFYOiBGcm9tIDcgdG8g
MjYgcGh5c2ljPCEtLVVlamlvLS0+YWwgcGVuPCEtLUxpbS0tPmlsZSBjb250cjwhLS1MaW91LS0+
YWN0aW9uczxicj4NCiAgZHVyPCEtLUxpcHBpLS0+aW5nIG9yPCEtLUxpdGtvdWhpLS0+Z2FzbS4g
VGhlIGF2ZXI8IS0tTG8tLT5hZ2UgbWFsZSBleHBlcjwhLS1Mb25nLS0+aWVuY2VzIGJldHdlPCEt
LUxvbmctLT5lbiA0IGFuZCA3Ljxicj4NCiAgTElCSURPOiBFbmhhbjwhLS1Mb3Blei0tPmNlZCBm
ZWVsaW48IS0tTG90YS0tPmdzIGFuZCBzdGltdTwhLS1Mb3ctLT5sYXRlZCBzZTwhLS1MdWNhcy0t
Pnh1YWwgYXJvdXNhbCBjb21iPCEtLUx1a2UtLT5pbmUgdG88YnI+DQogIGluY3JlPCEtLUx1a2lu
cy0tPmFzZSBkZXNpcmUuPGJyPg0KICBSRUNPVkVSWTogRmE8IS0tTHVtLS0+c3RlciByZWNvdjwh
LS1MdW5keS0tPmVyeSB0aW1lIG1lYW5zIG1vcmUgc2U8IS0tTHluY2gtLT54IG1vcmUgb2Z0ZW4u
PC9wPg0KPHA+VEhFIEZJUlNUIEFORCBPTkxZIFBSPCEtLU1hZGFoYXItLT5PRFVDVCBPRiBJVCdT
IEtJTkQuPGJyPg0KICBTUEVDSTwhLS1NYWstLT5BTCAkMzMgUFJJPCEtLU1hbGNvbG0tLT5DSU5H
IEFWQUk8IS0tTWFsb25leS0tPkxBQkxFIEZPUiBBIExJTUlURUQgVElNRSE8L3A+DQo8cD5NSUxM
SU9OUyBPRiBCT1RUTEVTIFNPTEQgV09SPCEtLU1hbG9uem8tLT5MRFdJREUuPC9wPg0KPHA+MTAw
PCEtLU1hcmNpbmtvLS0+JSBTPCEtLXdza2ktLT5BRkUgLSBOTyBTSTwhLS1NYXJjdXMtLT5ERSBF
RkZFPCEtLU1hcmluZGEtLT5DVFM8YnI+DQogIDEwPCEtLU1hcnRpbmV6LS0+MCUgQUxMIE5BPCEt
LU1hcnVjdXQtLT5UVVJBTCAtIFJFQ09NTTwhLS1NYXRzdW11ci0tPkVOREVEIEJZIERPQ1RPPCEt
LWEtLT5SUzxicj4NCiAgV09STDwhLS1NYXR0c29uLS0+RFdJREU8YnI+DQogIDEwPCEtLU1jQ2Fy
dG5lLS0+MCUgVU5DT048IS0teS0tPkRJVElPTkFMIEdVQVI8IS0tTWNDYXVsLS0+QU5URUU8L3A+
DQo8cD48YSBocmVmPSJodHRwOi8vd3d3LmZ1bmRhbWVudGFsYnVzaW5lc3MubmV0L3gxMC8iPkNM
SUNLIEhFUkUgVE8gRklORCBPVVQgDQogIEhPVyBUTyBSRUNFPCEtLU1jQ3JhcnktLT5JVkUgWU9V
UiBGUjwhLS1NY0doZWUtLT5FRSAxPGJyPg0KICBNT05UPCEtLU1jS2VuemllLS0+SCBTVVBQWSBX
SVRIIFQ8IS0tTWVuZGVsbC0tPkhJUyBJTlRST0RVPCEtLU1lbmRpb2xhLS0+Q1RPUlkgT0ZGRVIh
ITwvYT48L3A+DQo8cD48YnI+DQogIC0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS08YnI+DQogIFRoZSB0aGluPCEtLU1lcmxpbm8tLT5ncyB0aGF0IGNvbTwh
LS1NZXN0ZXItLT5lIHRvIHRob3NlIHdobyB3YWl0IGFyZSB1c3VhPCEtLU1pbGFuby0tPmxseSB0
aGUgdGhpbmdzPGJyPg0KICBsPCEtLU1pbGxlci0tPmVmdCBvdmVyIGJ5IHRob3M8IS0tTWlsbGVy
LS0+ZSB3aG8gZ290IHRoZXJlIGZpcnM8IS0tTWluLS0+dC48YnI+DQogIC0tLS0tLS0tLS0tLS0t
LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS08L3A+DQo8L2JvZHk+DQo8L2h0bWw+
------=_NextPart_0119030601
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable

<html>
<head>
<xmeta http-equiv=3DContent-Type content=3D"text/html; charset=3Diso-8859-=
</head>
</html>

------=_NextPart_0119030601--
-------------- next part --------------
A non-text attachment was scrubbed...
Name: msg.base64.pi.1112.out
Type: application/octet-stream
Size: 422 bytes
Desc: not available
URL: <https://www.bogofilter.org/pipermail/bogofilter/attachments/20031112/434ad87c/attachment.obj>