fatal flex scanner internal error--end of buffer missed

[Gyepi SAM]

On Sat, Jan 11, 2003 at 12:20:42AM -0500, David Relson wrote:
> At 12:13 AM 1/11/03, you wrote:
> >> Do you think we need to pursue this?
> >
> >Yes, since the original problem still exists. The base64 decoder should
> >fail gracefully on undecodable input.
> 
> You seem to have an approach in mind for handling the problem.  I 
> don't.  Would you care to take the lead on this one?  Do you want/need 
> another copy of the message?

OK. The simplest fix I have found is the attached patch which returns
the count so far as soon as we detect that we'll generate a negative return
value.  It correctly handles the input that caused the failure.
Two possible issues that came up:

1. If a line contains invalid input followed by valid input, we'll never
   get to decode the valid input
2. If the invalid input is in the base64 alphabet, we decode it, even if the
   results are meaningless.

I don't think neither of these should be cause for concern since in either case,
the base64 input is probably sporked anyway. The concern here is that invalid input be handled gracefully, and the patch does that.

-Gyepi  
-------------- next part --------------
Index: base64.c
===================================================================
RCS file: /cvsroot/bogofilter/bogofilter/base64.c,v
retrieving revision 1.1.2.1
diff -B -u -b -r1.1.2.1 base64.c
--- base64.c	29 Dec 2002 21:57:49 -0000	1.1.2.1
+++ base64.c	11 Jan 2003 05:39:19 -0000
@@ -57,6 +57,8 @@
 	    byte t = base64_xlate[c];
 	    if (t == base64_invalid) {
 		shorten = 4 - i;
+		if (shorten > count)
+			return count;
 		i = 4;
 		v >>= (shorten * 2);
 		if (shorten == 2) s++;