AW: AW: Why do I still need MacToISO, when working with UTF-8?

[Tiemo Hollmann TB]

Hi Mark,
thank you for taking your time and clarifying. I wasn't aware that the
internal format on a Mac client is MacRoman. I thought it would be a
"neutral" UTF-8 format.
Thanks
Tiemo


-----Ursprüngliche Nachricht-----
Von: use-livecode [mailto:use-livecode-bounces at lists.runrev.com] Im Auftrag
von Mark Waddingham via use-livecode
Gesendet: Montag, 16. Januar 2017 17:42
An: How to use LiveCode <use-livecode at lists.runrev.com>
Cc: Mark Waddingham <mark at livecode.com>
Betreff: Re: AW: Why do I still need MacToISO, when working with UTF-8?

Hi Tiemo,

Okay so, I'm assuming that all this code is running on the Mac client...

> *put fld "name" into myName*

At this point myName contains a (text) string - thus encoding issues don't
exist (you should think of text strings in memory as being stored in an
'encoding neutral' format).

> *open file myFile for binary write*
> *write myName to file myFile*
> *close file myFile*

This piece of code will open a file on disk in the native encoding of the
platform - so MacRoman. It will convert this from the internal encoding to
MacRoman on writing. Thus your text file will be a MacRoman encoded text
file.

> *open file myFile for binary read*
> *read from file myFile until EOF*
> *close file myFile*
> *put it into myName*

This piece of code will read from a file on disk and assume that it is in
the native encoding of the platform - so, in this case, MacRoman. It will
convert the content of the file from that to the internal encoding.

Up to this point - because you saved and loaded the file on the same
platform the content of myName should be as you expect -- unchanged.

> *if the platform is "MacOS" then put macToISO(theName) into theName*

When run on Mac this line will execute and do the following:

    1) Convert theName to a binary string - this uses the native platform
encoding (MacRoman)
    2) Map each byte from the MacRoman code index to the ISO Latin-1 code
index

This essentially converts theName from a text string to a binary string
encoded in Latin-1.

> *put URL ("http://myUser:myPW@myURL" & "mySQL.php?" &
> URLEncode(theName))
> into rslt*

This line constructs the URL - it is making the assumption that PHP (at the
other end) will interpret the bytes after the '?' as representing
Latin-1 encoded text.

> Without macToISO on a Mac client theName enters corrupted in the mySQL 
> db

This is most likely because PHP is defaulting to 8859-1 or Latin-1 as the
encoding used in URLEncoded fields in a URL. If you don't do MacToIso, then
you will be passing up MacRoman encoded text (URLencoded) to PHP, which can
happily be decoded as Latin-1 or 8859-1 (Latin-1 is a superset of 8859-1),
but with some chars (such as accented letters) in different places.

What you need to do here is explicitly UTF8 encode theName before passing it
to URLEncode, then explicitly decode it as UTF8 on the PHP side (or set a
property in PHP which changes the default assumption about URLs - I
apologise for not being more accurate here, my knowledge of PHP is a little
stale these days!).

Warmest Regards,

Mark.

--
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps

_______________________________________________
use-livecode mailing list
use-livecode at lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode