Uniencode fails on some chars

[Tuviah M Snyder]

>You are right about these characters. I used the moji palette to get the
>codes that I reported. I guess the problem I see is that uniencode just
>stops at that point rather than skipping it and moving on. Maybe my hope
>that it would at least ignore it and continue is unrealistic.
Please send a sample file, and instructions for reproducing this to
tuviah at runrev.com 

>BTW did you ever get a response to the difference between SJIS utf-8,
and
>utf-16?
UTF-16 files usually have a byte order mark at the beginning (to tell the
parse whether to interpret as big endian or small endian) so you can
usually tell if a file is unicode by looking for this byte order mark at
the beginning of a UTF-16 file.

see: http://www.wikipedia.org/wiki/Byte_Order_Mark

Now telling the difference between utf-8 and shift-JIS is much more
difficult. You have to go through the actual text and look for illegal
encodings, and even then you can't really be sure.

Tuviah Snyder <tuviah at runrev.com> <http://www.runrev.com/>
Runtime Revolution Limited - Software at the Speed of Thought