NUL/CR munging in -p mode fixed -- time for 0.7.6? (was: problem with stripping NULs from input?)

[Matthias Andree]

-----BEGIN PGP SIGNED MESSAGE-----

On Tue, 22 Oct 2002, Allyn Fratkin wrote:

> i submitted a patch a day or so ago to modify the lexer's use of
> fgets to replace NUL characters with spaces.  the problem being solved
> is that most string routines, including fgets, treat a NUL as the end
> of the string.  anything past a NUL was ignored.  worse, bogofilter treated
> a line that started with NUL as the end of file and stopped reading further.
> 
> i don't use -p mode so this didn't occur to me before. the lexer is now
> modifying the input in such a way that it cannot be properly output
> if the -p option is in use.  NULs are turned to spaces.
> 
> how serious is this in the short term?  it seems as though this
> will take a bit of effort to overcome and fix.

It is critical to data integrity. Thanks for bringing this up.

It was not really as big an effort as you thought, because the lexer
uses buffer & len rather than NUL-terminated strings, and there are only
two places that treat the "passthrough" texts: lexer.l, when storing,
and main.c, when priting. So the important part was to store the
unmodified input.

It took adding a length field to the text blocks, shuffling the \r\n -> \n
mapping to after the textblocks are stored and dropping the NUL -> SPC
mapping.

To achieve this, I changed the broken return semantics of the fgets
replacement to return the count of characters instead, the rest was
trivial, replacing strcpy by memcpy and fputs by fwrite.

I fixed some other bugs along the way, see the CVS Log.

IMHO, it's time for 0.7.6 beta. What do you and the others think?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iQCVAwUBPbilwydEoB0mv1ypAQF+HQP/US97ARAnt+kqaQzInY1aKWpg93LUcYJ8
0a/bkvM65K457k5jxsYno+I7myl09l4jPtA79hsaVlMD3s/P7KjHuVn1pDo/8Gv4
xv/lFBmPiFZo0ZoezCmyk/nur0t0wnUu7fOYlEMjUge6929t6JayRoSxC2vIzMUD
hL5uir0Y/Ww=
=o5HD
-----END PGP SIGNATURE-----