AW: AW: Why do I still need MacToISO, when working with UTF-8?

[Sébastien Nouat]

Hi Tiemo,

As an additional note (if you don't absolutely need to write binary) it 
is also possible to use the syntax

/open file <file> for "utf8" text read/

or

/open file <file> for "utf8" text write/


in which cases the engine takes care of encoding/decoding the string 
using UTF-8 encoding.

Then, when calling

/write <string> to file <file>/

/string/ can be a LiveCode string with Unicode characters, and will be 
written in UTF-8, and

/read from file <UTF-8-encoded file> for 1 lines

/will set /it/ to a string with the decoded UTF-8 string.


Regards,

Sebastien

On 16/01/2017 17:14, Tiemo Hollmann TB via use-livecode wrote:
> Hi Mark,
> thank you for taking your time and clarifying. I wasn't aware that the
> internal format on a Mac client is MacRoman. I thought it would be a
> "neutral" UTF-8 format.
> Thanks
> Tiemo
>
>
> -----Ursprüngliche Nachricht-----
> Von: use-livecode [mailto:use-livecode-bounces at lists.runrev.com] Im Auftrag
> von Mark Waddingham via use-livecode
> Gesendet: Montag, 16. Januar 2017 17:42
> An: How to use LiveCode <use-livecode at lists.runrev.com>
> Cc: Mark Waddingham <mark at livecode.com>
> Betreff: Re: AW: Why do I still need MacToISO, when working with UTF-8?
>
> Hi Tiemo,
>
> Okay so, I'm assuming that all this code is running on the Mac client...
>
>> *put fld "name" into myName*
> At this point myName contains a (text) string - thus encoding issues don't
> exist (you should think of text strings in memory as being stored in an
> 'encoding neutral' format).
>
>> *open file myFile for binary write*
>> *write myName to file myFile*
>> *close file myFile*
> This piece of code will open a file on disk in the native encoding of the
> platform - so MacRoman. It will convert this from the internal encoding to
> MacRoman on writing. Thus your text file will be a MacRoman encoded text
> file.
>
>> *open file myFile for binary read*
>> *read from file myFile until EOF*
>> *close file myFile*
>> *put it into myName*
> This piece of code will read from a file on disk and assume that it is in
> the native encoding of the platform - so, in this case, MacRoman. It will
> convert the content of the file from that to the internal encoding.
>
> Up to this point - because you saved and loaded the file on the same
> platform the content of myName should be as you expect -- unchanged.
>
>> *if the platform is "MacOS" then put macToISO(theName) into theName*
> When run on Mac this line will execute and do the following:
>
>      1) Convert theName to a binary string - this uses the native platform
> encoding (MacRoman)
>      2) Map each byte from the MacRoman code index to the ISO Latin-1 code
> index
>
> This essentially converts theName from a text string to a binary string
> encoded in Latin-1.
>
>> *put URL ("http://myUser:myPW@myURL" & "mySQL.php?" &
>> URLEncode(theName))
>> into rslt*
> This line constructs the URL - it is making the assumption that PHP (at the
> other end) will interpret the bytes after the '?' as representing
> Latin-1 encoded text.
>
>> Without macToISO on a Mac client theName enters corrupted in the mySQL
>> db
> This is most likely because PHP is defaulting to 8859-1 or Latin-1 as the
> encoding used in URLEncoded fields in a URL. If you don't do MacToIso, then
> you will be passing up MacRoman encoded text (URLencoded) to PHP, which can
> happily be decoded as Latin-1 or 8859-1 (Latin-1 is a superset of 8859-1),
> but with some chars (such as accented letters) in different places.
>
> What you need to do here is explicitly UTF8 encode theName before passing it
> to URLEncode, then explicitly decode it as UTF8 on the PHP side (or set a
> property in PHP which changes the default assumption about URLs - I
> apologise for not being more accurate here, my knowledge of PHP is a little
> stale these days!).
>
> Warmest Regards,
>
> Mark.
>
> --
> Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps
>
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>
>
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode