What is a word (lexertest)

[Boris 'pi' Piwinger]

Hi!

Even though I don't code here, I tested something;-)

[3.14 at pi ~/local/bogolists]$ echo "»cmsg newgroup«"|lexertest
get_token: 1 '»cmsg'
get_token: 1 'newgroup«'
[3.14 at pi ~/local/bogolists]$ echo "bla"|lexertest

Both results are not really satisfiying. There might be a reason why
the second does not return anything, but the first is wrong. Well,
here we have the problem that we cannot tell without looking at the
charset.

pi