Accented Characters Display

[Devin Asay]

On Apr 7, 2011, at 5:40 PM, Mark Schonewille wrote:

> Hi,
> 
> The whole idea of using unicode is that you don't need isoToMac anymore.
> 
> If you happen to be storing unicode data in a database that isn't encoded as unicode, then it simply won't work. What is the encoding of the database? Is it actually UTF8?
> 
> There's a good chance that LiveCode is unable to retrieve the unicode data directly from the MySQL database, because LiveCode cannot retrieve binary data from the command line, externals and (database) drivers. LiveCode can, however, retrieve binary data through sockets. If you make a simple PHP script, which shold reside on the server and can be called by LiveCode using the put URL command, then you will be able to retrieve correct unicode data.
> 
> Once you got the PHP script, you should be able to execute a simple script like this:
> 
> put "http://domain.com?query=read&name=john&city=new%20york" into myUrl
> put url myUrl into myData
> put uniencode(myData,"UTF8") into myData
> set the unicodeText of fld "Foo" to myData
> 
> To save the data you need to do something like this:
> 
> put urlEncode(unidecode(the unicodeText of fld "Name","UTF8")) into myName
> put urlEncode(unidecode(the unicodeText of fld "City","UTF8")) into myCity
> put "http://domain.com?query=write&name=" & myName & "&city=" & myCity into myUrl
> get url myUrl
> 
> This is just an example to inspire you. The above won't work with your set-up.
> 
> I delivered a project yesterday, which is based on this mechanism: always convert your data to UTF8 before saving it and convert it back to unicode (UTF16) when retrieving it. Make sure that the database is UTF8 capital insensitve encoded.
> 
> Why store as UTF8 and not as UTF16? Because you don't want to mess with little and big endian.

I'll second what Mark said. Storing your text as UTF-8 in the database is the way to go. I do this a lot, and it's been completely reliable. (But make sure to set your MySQL table encodings to UTF-8, as Mark indicated.

To save your data to the DB as UTF-8, take your text from a field with unicode text in it:

put unidecode(the unicodeText of fld "myUniFld","UTF8") into tUni8Txt

Then just execute a standard SQL statement (or send it through a server-side script) to insert it into your database table.

When you retrieve it from the DB, just reverse the process:

set the unicodeText of fld "myUniFld" to uniEncode(tTextFromDB,"UTF8")

HTH

Devin

Devin Asay
Humanities Technology and Research Support Center
Brigham Young University