best/fastest way to tell if a field contains unicode text?

Fraser Gordon fraser.gordon at runrev.com
Fri Mar 21 06:56:39 EDT 2014


On 21 Mar 2014, at 01:36, Mark Wieder <mwieder at ahsoftware.net> wrote:
> 
> Thanks. Most of that makes sense, but why not alias the deprecated
> uni* functions to text* functions rather than just having them return
> the wrong thing?

UniEncode would alias to textDecode and uniDecode to textEncode as the closest equivalents. Unfortunately, they're not exactly equivalent - as I found out when I initially re-implemented them in 7.0. That's the reason behind most of the functions we've marked as deprecated in 7.0 - upgrading them to support Unicode would break existing stacks in subtle ways (e.g. numToChar - somebody, somewhere, might be depending on the fact that it interprets the number modulo 256. Plus, it takes the native encoding rather than Unicode codepoints).

> 
> The textEncode function does indeed work as expected. I hadn't checked
> out the release notes before trying this.

Reading the release notes is almost as common as reading the (friendly?) manual… It does make me wonder if one of the most important pieces of feedback from this DP is that we need to find some way to better highlight these changes. Somewhat ironically, the transparent Unicode support means:

- New stacks don't require any additional work to support Unicode
- Existing stacks with no Unicode support need (hopefully) minor changes where they do binary I/O
- Existing stacks using uni* should remove all the uni* stuff

Thankfully, removing uni* doesn't need to be done all at once - if you set the unicodeText of a field and then get the text of it, the text you'll get will be Unicode. The main disadvantage to continuing to use the unicodeText is that some of the new chunk expression features may not work properly (e.g. trueWord won't work; char won't properly account for combining characters, etc) because they are operating on binary data rather than text.

Regards,
Fraser



More information about the use-livecode mailing list