replace_nonascii_characters [was: using iconv()]

[David Relson]

On Mon, 10 Jan 2005 01:50:04 +0300
Evgeny Kotsuba wrote:

...[snip]...


> For some non-english speakers replaing nonascii - characters is  also 
> very good thing for the same reasons but for asians codepages, or more 
> correct - for codepages with allow_nonascii_replacement.  In all 
> internet software russians almost automatically set 
> "replace_nonascii_characters=false", "allow 8bit coding" and so on.
> In case  replace_nonascii_characters=false
> 
> if ( replace_nonascii_characters && charset->allow_nonascii_replacement)

Perhaps charsets need additional attributes (beyond the present
allow_nonascii_replacement)....


> will be always false...
> 
> By the way, why substitue '?' not just space ?

Using the '?' (which is allowed in tokens) causes the asian spam to
produce tokens that can be used in scoring.  Using space would cause the
message body to be ignored.