Unicode is not "everywhere"...

[Paul Dupuis]

I just want it consistent and documented and able to return more than 
just ASCII data

Currently, OSX shell returns UTF8 which may mean that it is returning 
binary as it is returning 8-bit bytes where Unicode text has been 
encoded as UTF8

Windows returns CP1252 text, not binary and any Unicode results, which 
DOS displays as Unicode just fine, can be returned without elaborate 
work-arounds.

That by definition is a bug.

I would advocate that shell should return binary data. Text being 
returned should be UTF8 encoded, that way people expecting ASCII do nto 
need to o anything, they can just work with teh returned text. People 
expecting Unicode can use textDecode to get the UTF8 converted to LC 
native 16-bit Unicode, and people extcting binary can use the byte chunk 
to process what comes back however they want.

The current missing part is the UTF8 encoding of text being returned 
under Windows.

Also, as my platforms of concern are Windows and OSX, I have not tested 
shell() under Linux. I assume it behaves like OSX but that is an assumption.


On 8/22/2019 8:36 PM, Dar Scott Consulting via use-livecode wrote:
> You can't have it both ways.
>
> Either shell() only characters of the broadest character set, Unicode, or it returns binary. If it returns characters, then it will not work for binary responses; that will have to be documented as excluded.
>
> If Unicode is returned then textDecode is not needed and even should not be used.
>
>
>
>> On Aug 22, 2019, at 2:08 PM, Richard Gaskin via use-livecode <use-livecode at lists.runrev.com> wrote:
>>
>> Dar Scott wrote:
>>
>>> If the result of shell() is run through textDecode, should then
>>> shell() return binary data? This puts it outside the scope of
>>> "everywhere". I like the idea, as long as simple ASCII characters
>>> work transparently.
>> Isn't that the goal of UTF-8, which is why so many programming languages uses it as the default?
>>
>> -- 
>> Richard Gaskin
>> Fourth World Systems
>> Software Design and Development for the Desktop, Mobile, and the Web
>> ____________________________________________________________________
>> Ambassador at FourthWorld.com                http://www.FourthWorld.com
>>
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode
>