Merge and unicode

[Dar Scott Consulting]

I think you are trying to think too much about the LC implementation of text. Maybe.

Text in LC is an abstraction of a sequence of code points. Whether it is UTF16 or not is hidden to me. (mostly) 

So,

    get textDecode( binaryFromServer, "UTF-8" )

should put that into the correct form, if it is really UTF-8.

A data (binary bytes) is interpreted as native encoding if one tries to use it as text. I recommend against this. I try to always textDecode() everything coming in, but I make exceptions at times for ASCII. 

I'm not sure what you mean by metadata. Are you referring to HTTP content-type?

Sorry, if I am off on a bunny trail...

Dar

> On Sep 9, 2019, at 4:38 PM, J. Landman Gay via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> It's UTF8 text from a server, which I textDecode to UTF16. When I use the UTF16 text in a merge, diacriticals and/or curly quotes get mangled. (Same with setting metadata on field text too.)
> 
> On 9/9/19 4:16 PM, Dar Scott Consulting via use-livecode wrote:
>> I'm not sure I understand.
>> Do you mean "encoded to UTF-16"? In that case you should decode that to convert it to internal text. And then try merge. (Which still might have problems, I suppose.)
>>> On Sep 9, 2019, at 12:08 PM, J. Landman Gay via use-livecode <use-livecode at lists.runrev.com> wrote:
>>> 
>>> It seems that the merge command doesn't respect unicode. Does anyone have a workaround? The text I'm inserting is already decoded to UTF16.
>>> 
>>> -- 
>>> Jacqueline Landman Gay         |     jacque at hyperactivesw.com
>>> HyperActive Software           |     http://www.hyperactivesw.com
>>> 
>>> _______________________________________________
>>> use-livecode mailing list
>>> use-livecode at lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> 
> -- 
> Jacqueline Landman Gay         |     jacque at hyperactivesw.com
> HyperActive Software           |     http://www.hyperactivesw.com
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode