replace-nonascii-characters stopped working?

[David Relson]

On Mon, 21 Mar 2005 08:31:55 -0600
Bill McClain wrote:

> I believe that replace-nonascii-characters stopped working sometime
> between 0.93.4 and 0.94.1. Dumping the wordlist I see 8-bit chars in
> recently registered tokens, which are also visible using "bogofilter
> -vvv" on an appropriate message. I'm pretty sure these used to be
> replaced by "?".
> 
> -Bill

Hi Bill,

It's working fine here.  I tested using:

[relson at osage src]$ sh -x rnc.sh
+ for YN in yes no
+ bogolexer -C --replace-nonascii-characters=yes -H -p -I msg.rnc.txt -O rnc-yes.out
+ for YN in yes no
+ bogolexer -C --replace-nonascii-characters=no -H -p -I msg.rnc.txt -O rnc-no.out
+ ls -l rnc-no.out rnc-yes.out
-rw-r--r--  1 relson relson 1421 Mar 21 18:40 rnc-no.out
-rw-r--r--  1 relson relson 1421 Mar 21 18:40 rnc-yes.out
+ md5sum rnc-no.out rnc-yes.out
ea5c2c38500236bfee1ae5a58d0aa945  rnc-no.out
558665fc2779a8acc0e2cc16fb5e815f  rnc-yes.out

Attached is a tarball with test script, input file, and the two output
files.  Let me know what it does on your machine.

Now the question now becomes:  what environment are your running in and
what, exactly are you doing?

Regards,

David

-------------- next part --------------
_______________________________________________
Bogofilter mailing list
Bogofilter at bogofilter.org
http://www.bogofilter.org/mailman/listinfo/bogofilter