eastern language support [was: Re: [bogofilter-announce] Version 0.7.5 Released]

[Boris 'pi' Piwinger]

Matthias Andree <matthias.andree at gmx.de> wrote:

>> are there plans to add stop characters from other charsets, and for that
>> matter, allow the lexer to recognize different charsets?
>
>Well, the US-ASCII supersets (many ISO-8859 variants) should be covered,
>ISO-8859-2 should be no problem AFAICS. 

The problem is that some codes are alpha-characters in one
and not in another of those. » is a quote character in
ISO-8859-1, but an alpha-character in ISO-8859-2. So the
solution is (to repeat an earlier discussion) to convert
into Unicode first.

>UTF-8 with Asian languages might be.

I guess so. But I really don't understand that business. Do
we have someone here who speaks any of those languages?

pi