What has become of buff and word and fgetsl?

[David Relson]

At 10:47 AM 2/25/03, Nick Simicich wrote:
>At 08:52 PM 2003-02-24 +0100, Matthias Andree wrote:
>>The former xfgets code used to append data to the buffer (I don't see
>>where that's used) and return the amount of octets read. I have changed
>>the current xfgets code to stomp over whatever is left in the buffer
>>from a previous call. This passes "make check", so I'll commit right
>>away, without further questions. I've now fixed the callee. Anything
>>else will have to be fix in the callers.
>
>I *think* that the buffer passed to yyinput or whatever is actually not at 
>the beginning.  There is a case where the buffer is to be appended to when 
>text is put off and then bubbled back up.  This is where I have been 
>tracing for the past two days.
>
>The buffer as passed by lex is some offset from the actual buffer.  I have 
>no idea if there used to be a need to know where in the buffer you were, 
>but lex figures where in the buffer it needs the next input.

Right.  It's especially noticeable with Greg's n*100k 'x' files, which 
bogofilter used to only partially process.  Stepping through the code, I 
could see flex ask for 8192 bytes, get 76, ask for 8192-76, get 76 more, 
ask for 8192-76-76, etc.  Changing address and length is a classic method 
for adding more data to an existing buffer.  Bogofilter's code for killing 
html comments uses the same technique to process multiline commends.