[cvs] bogofilter/src convert_unicode.c,1.5,1.6

[David Relson]

On Mon, 20 Jun 2005 00:54:06 +0200
Matthias Andree wrote:

> David Relson <relson at users.sourceforge.net> writes:
> 
> > Update of /cvsroot/bogofilter/bogofilter/src
> > In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2292
> >
> > Modified Files:
> > 	convert_unicode.c 
> > Log Message:
> > Use identity map when invalid conversion is specified.
> >
> > Index: convert_unicode.c
> > ===================================================================
> > RCS file: /cvsroot/bogofilter/bogofilter/src/convert_unicode.c,v
> > retrieving revision 1.5
> > retrieving revision 1.6
> > diff -u -d -r1.5 -r1.6
> > --- convert_unicode.c	18 Jun 2005 13:48:10 -0000	1.5
> > +++ convert_unicode.c	18 Jun 2005 13:51:28 -0000	1.6
> > @@ -128,9 +128,11 @@
> >      cd = iconv_open( to_charset, from_charset );
> >      if (cd == (iconv_t)(-1)) {
> >  	int err = errno;
> > -	if (err != EINVAL)
> > -	    fprintf( stderr, "Invalid charset '%s'\n", to_charset );
> > -	cd = iconv_open( from_charset, from_charset );
> > +	if (err == EINVAL) {
> > +	    /* error - use identity mapping */
> > +	    fprintf( stderr, "Conversion from '%s' to '%s' is not supported.\n", from_charset, to_charset );
> > +	    cd = iconv_open( "iso-8859-1", "iso-8859-1" );
> 
> Are you sure this is transparent for characters in the 0x80 - 0x9f and
> perhaps the 0x00 - 0x1f ranges? I'm not. I'm not even sure bogofilter
> will then find the same tokens if they happen to be seen again in a
> different character set - we might rather want to either reject the
> whole registration.

Perhaps we'd do better if we disable translation when iconv_open()
rejects the character set ???