Merge and unicode

Dar Scott Consulting dsc at swcp.com
Mon Sep 9 22:58:04 EDT 2019


This quick check seems to work for me.

on mouseup

put "A" into field 1

set the metadata of char 1 of field 1 to "é"

put the metadata of char 1 of field 1 after field 1

end mouseup


> On Sep 9, 2019, at 8:32 PM, J. Landman Gay via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Well, I've made some changes to the code since I started urlEncoding the text before merging so I'll check that again. Paul is right that unicode in htmltext needs to be in hex, but the numbers I'm getting back are very high (8,000+) and render in the field as strange pictographs. Elsewhere where there is no merge, curly quotes translate to the named quote or apostrophe entities and are correct.
> 
> By metadata I mean the LC term (see the dictionary) that allows you to attach some text to a field text chunk. The metadata isn't displayed in the field but you can use it for anything you want. In my case the field is a list of clickable entries in a table of contents, each with its own metadata attached that provides a path to the stack and card the entry needs to open.
> 
> When I use normal LC text as metadata, diacriticals aren't rendered correctly (curly quotes become question marks,) the path is therefore incorrect and the click goes nowhere.
> 
> Since LC is supposed to be unicode throughout, I'd expect metadata to be compatible. The same text appears correctly when not used as metadata.
> --
> Jacqueline Landman Gay | jacque at hyperactivesw.com
> HyperActive Software | http://www.hyperactivesw.com
> On September 9, 2019 7:25:28 PM Dar Scott Consulting via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
>> I think you are trying to think too much about the LC implementation of text. Maybe.
>> 
>> Text in LC is an abstraction of a sequence of code points. Whether it is UTF16 or not is hidden to me. (mostly)
>> 
>> So,
>> 
>> get textDecode( binaryFromServer, "UTF-8" )
>> 
>> should put that into the correct form, if it is really UTF-8.
>> 
>> A data (binary bytes) is interpreted as native encoding if one tries to use it as text. I recommend against this. I try to always textDecode() everything coming in, but I make exceptions at times for ASCII.
>> 
>> I'm not sure what you mean by metadata. Are you referring to HTTP content-type?
>> 
>> Sorry, if I am off on a bunny trail...
>> 
>> Dar
>> 
>>> On Sep 9, 2019, at 4:38 PM, J. Landman Gay via use-livecode <use-livecode at lists.runrev.com> wrote:
>>> 
>>> It's UTF8 text from a server, which I textDecode to UTF16. When I use the UTF16 text in a merge, diacriticals and/or curly quotes get mangled. (Same with setting metadata on field text too.)
>>> 
>>> On 9/9/19 4:16 PM, Dar Scott Consulting via use-livecode wrote:
>>>> I'm not sure I understand.
>>>> Do you mean "encoded to UTF-16"? In that case you should decode that to convert it to internal text. And then try merge. (Which still might have problems, I suppose.)
>>>>> On Sep 9, 2019, at 12:08 PM, J. Landman Gay via use-livecode <use-livecode at lists.runrev.com> wrote:
>>>>> 
>>>>> 
>>>>> It seems that the merge command doesn't respect unicode. Does anyone have a workaround? The text I'm inserting is already decoded to UTF16.
>>>>> 
>>>>> 
>>>>> --
>>>>> Jacqueline Landman Gay         |     jacque at hyperactivesw.com
>>>>> HyperActive Software           |     http://www.hyperactivesw.com
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> use-livecode mailing list
>>>>> use-livecode at lists.runrev.com
>>>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>> _______________________________________________
>>>> use-livecode mailing list
>>>> use-livecode at lists.runrev.com
>>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>> 
>>> 
>>> --
>>> Jacqueline Landman Gay         |     jacque at hyperactivesw.com
>>> HyperActive Software           |     http://www.hyperactivesw.com
>>> 
>>> _______________________________________________
>>> use-livecode mailing list
>>> use-livecode at lists.runrev.com
>>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>> 
>> 
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
> 
> 
> 
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode




More information about the use-livecode mailing list