bug: "invalid buffer size" with current CVS

[David Relson]

At 05:56 AM 1/31/03, Matthias Andree wrote:

>On Thu, 30 Jan 2003, Matt Armstrong wrote:
>
> > Current CVS fails to parse the message
> > http://www.lickey.com/~matt/bogo/bufsize.msg.gz with an "Invalid
> > buffer size, exiting." error.  It is a 10k download.
>
>I can reproduce this.


Matthias,

Below are the first few lines from bogolexer (verbose/debug mode).  It's a 
base64 encoded bitmap and function yy_get_next_buffer() in the flex code 
(not our code) keeps asking for more data until it runs out of buffer space.

The interesting line in bogolexer's output is "Unknown mime type - 
'image/bmp;'".  Seems like at one time we were ignoring attachments.  Seems 
like that might be the right thing to do for images as bogofilter isn't 
going to find any meaningful text inside them.  This may take the bulk of 
the message away from bogofilter's scanning, but bogofilter _does_ still 
socre the headers for spamicity.

What's your take on this sitation?

David

[relson at osage cvs]$ bogolexer -x m -vvvv < ../msg.d/msg.ma.0131.txt
normal mode.
*** mime_reset
*** mime_push. stackp: 0
*** mime_type: Content-Type: multipart/related;
*** --> mime_boundary_set: 0 
'boundary="----=_NextPart_000_00BB_01C2C859.93FD70E0"'
*** <-- mime_boundary_set: 0 '----=_NextPart_000_00BB_01C2C859.93FD70E0'
*** got_mime_boundary:  stackp: 0, boundary: 
'------=_NextPart_000_00BB_01C2C859.93FD70E0'
*** mime_push. stackp: 1
*** mime_type: Content-Type: multipart/alternative;
*** --> mime_boundary_set: 1 
'boundary="----=_NextPart_001_00BC_01C2C859.93FEF780"'
*** <-- mime_boundary_set: 1 '----=_NextPart_001_00BC_01C2C859.93FEF780'
*** got_mime_boundary:  stackp: 1, boundary: 
'------=_NextPart_001_00BC_01C2C859.93FEF780'
*** mime_push. stackp: 2
*** mime_type: Content-Type: text/plain;
*** mime_encoding: Content-Transfer-Encoding: quoted-printable
*** mime_decode 1 ""
*** mime_decode 44 "------=_NextPart_001_00BC_01C2C859.93FEF780"
*** got_mime_boundary:  stackp: 2, boundary: 
'------=_NextPart_001_00BC_01C2C859.93FEF780'
*** mime_pop. stackp: 2
*** mime_push. stackp: 2
*** mime_type: Content-Type: text/html;
*** mime_encoding: Content-Transfer-Encoding: quoted-printable
*** mime_decode 1 ""
*** mime_decode 46 "------=_NextPart_001_00BC_01C2C859.93FEF780--"
*** got_mime_boundary:  stackp: 2, boundary: 
'------=_NextPart_001_00BC_01C2C859.93FEF780--'
*** mime_pop. stackp: 2
*** got_mime_boundary:  stackp: 1, boundary: 
'------=_NextPart_000_00BB_01C2C859.93FD70E0'
*** mime_pop. stackp: 1
*** mime_push. stackp: 1
Unknown mime type - 'image/bmp;'
*** mime_encoding: Content-Transfer-Encoding: base64
*** mime_decode 77 
"Qk12qgMAAAAAADYAAAAoAAAAQAIAAIsAAAABABgAAAAAAECqAwDEDgAAxA4AAAAAAAAAAAAAQkFC"
*** mime_decode 77 
"QkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFC"
*** mime_decode 77 
"QkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFC"
*** mime_decode 77 
"QkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFCQkFC"