NUL termination [was: Lexer restructure.]

[David Relson]

At 09:12 PM 1/27/03, Matthias Andree wrote:


>As a side note, I noticed David "fixed" lexer stuff to NUL-terminate
>strings. I wonder if that's safe, because we need to pass on NUL bytes
>as far as possible -- instead of making low-level functions that do have
>a count compatible with C strings (which is bound to wreak havoc when
>NUL bytes are in the input because then strlen() doesn't match the
>count), the functions that use C strings need to be fixed to use fwrite
>and all that.

Matthis,

The lexer routines that work with the buffer use a byte count and don't 
stop on a NUL byte.  I added the NUL byte because index() was going past 
the end of the buffer and segfaulting.  My changes deliberately allow only 
size-1 bytes in the buffer so that the NUL byte can be stored at 
buffer[count] without going beyond the buffer.  An alternate solution would 
be to use a length controlled version of index().  I think that would be 
overkill in this situation.

David