LC7 & Unicode

[Paul Hibbert]

> No response to my prior post, so let's try this again ...

I'll give some answers as far as I understand LC7 at present, if I get something wrong no doubt someone will correct me! It wouldn't be the first time. :)

> Dr Peter Brett wrote: "Internally, the LC7 engine only uses Unicode if it
> has to.  If your application only uses native strings, then LC7 will only
> use native strings.  Built-in Unicode support has very little to do with the
> fact that LC7 is slower for some workload"
> 
> I don't now, and don't intend to, use Unicode.

You could try LC6.7.3 instead, as far as I'm aware it has most of the same features as LC7 apart from the native unicode.

> I don't want *ANY* overhead associated with a feature I don't use.

If you really need to use LC 7.x.x, then you currently have no choice, it should work fine with or without Unicode, it may be slower, but the engineers are working to improve that.

> How can we determine whether or not Unicode is being "used"?

I can't see any easy way to do that, but I seem to remember some people discussing it on this list a while ago, maybe somebody could chime in.

> How does the LC7 engine detect Unicode is being used?

I really don't know, but I'd guess it has some way to detect the number of bytes used per character.

> Can we turn off that detection, thus eliminating Unicode processing?

No, this has been asked several times on the list, it's part of the internal processing that LC uses and is too deeply embedded to eliminate.

> My situation: I create a "chunk" of data by concatenating binary data to
> "native strings".  Then, I save the chunk into a large roll-your-own
> one-record-at-a-time "database".  If LC7 is somehow automatically detecting
> Unicode, perhaps my "blob" of data is fooling it into thinking it's Unicode.

If there is a problem, this sounds like the type of case that the engineers are asking to see, if you can send your stacks to them and point out where the problem lies, they should be able to pinpoint the culprit and eliminate it from future builds. You would be helping them to build a better LiveCode, then you and the rest of the LC community benefits. If you are concerned with your app security, you can email the files to them, they will even sign an NDA agreement if necessary, but they do have a strong respect for customer confidentiality. If they never get to see your files, they may never see a similar problem, we all seem to generate unique situations (problems) from time to time.

> The "blob" also includes native HTML text.  Could that be involved somehow
> in Unicode detection?

If your HTML text contains some Unicode characters, maybe that is triggering the Unicode detection.

I hope you find a solution.

Paul