Importing unicode UTF8 text files - followup
J. Landman Gay
jacque at hyperactivesw.com
Wed Aug 19 22:29:04 CEST 2015
On 8/19/2015 2:34 PM, Peter TB Brett wrote:
> On 2015-08-19 21:09, J. Landman Gay wrote:
>> Just to follow up: When importing foreign language files, "file" isn't
>> working. Non-ascii characters show up as question marks and textDecode
>> does nothing.
>> Using "binfile" and then textDecoding does import correctly but then
>> text chunking by lines fails. Line endings are imported as 2 bytes
>> using byteToNum 13,10 (in that order) which isn't the line ending
>> standard for any OS. I have to replace those specifically.
>> These are text files saved as UTF-8, created on Mac OS X and imported
>> on Mac OS X. I'm using LC 7.0.6.
> Hmm, this all sounds quite problematic.
> Obvious thing to check: are you sure your files are valid UTF-8? The
> following command will print "0" if the file is valid UTF-8 and "1"
> iconv -f UTF-8 your_file > /dev/null ; echo $?
> Otherwise, if you have a test case that you don't mind sharing, could
> you please file a bug report and add me (e-mail address below) to the Cc
> list? Otherwise, please e-mail me directly. If you could test with
> LiveCode 7.0.1-rc-1, that would also be quite helpful.
> I *definitely can't* guarantee a quick fix, but it's possible something
> obvious is going wrong or we can find a quick workaround.
Thanks very much. The terminal command returns 0 (and BBEdit can handle
the file too) so it seems to be valid. I'll submit a bug report, I can
share the file without any issues.
Jacqueline Landman Gay | jacque at hyperactivesw.com
HyperActive Software | http://www.hyperactivesw.com
More information about the use-livecode