Unicode is not "everywhere"...

[Paul Dupuis]

Mark,

1) Fine with shell remaining as is. Thanks to Dar and others, there are 
ways to execute a windows or OSX shell command that returns Unicode and 
get Unicode back. The Dictionary entries just need to be documented 
with  a caution or warning about how to handle Unicode with Shell on 
each platform. I'd be happy to change the bug to a DOC bug and update 
the dictionary if you give the go-ahead.

2) Fine with urlEncode/urlDecode remaining as it. If LC want to make a 
set of functions urlEncodeRFC/urlDecodeRFC, I am all for that, however, 
just documenting the convention that Unicode should be UTF8 encoded 
before urlEncoding and UTF8 decoded after URLDecoding in the Dictionary 
would be sufficient. I opened two documentation bugs (one for each 
dictionary entry for urlencode/decode exactly for updating the 
Dictionary with the intent to do that myself as soon as I have a chance.

3) I (Researchware, Inc.) desperately needs a fix for the detailed 
files/detailed folder - asap. Our main application depends upon it. We 
have moved the app from LC6.7.11 to LC 9.0.5rc1 and made it full Unicode 
for the international market, but the part of our app that extensively 
manages users research documents for their projects relies on "the 
detailed files" which does not work with Unicode and my work-around 
using shell (see the bug entry) is PAINFULLY slow.

So new functions 'fileInfo' and 'folderInfo' would be wonderful or 
something - anything - that can be done sooner rather than later.

Researchware (between myself and Curry whose working for us) filed about 
20+ LC bugs in the last 60 days. The wonderful thing about LC is that 
we've been able to code work-arounds for most all of those many bugs. 
I've even coded a work-around for the "detailed" files, but it's 
performance due to the multiple shell calls is very poor. Our customers 
won't stand for it for very long, hence the desperate need for a 
solution sooner rather than later.

Please, please consider moving a solution to the detailed files to a 
forthcoming release.


On 8/27/2019 8:11 AM, Mark Waddingham via use-livecode wrote:
> On 2019-08-22 21:07, Paul Dupuis via use-livecode wrote:
>> I reported what I thought was 3 bugs in 1 report in
>> https://quality.livecode.com/show_bug.cgi?id=22213. I have edited that
>> report to focus on a single bug - that the detailed files (and
>> probably the detailed folders) is broken for Unicode as every Unicode
>> character in a file name is encoded as %3F or ?. Originally I had
>> though that meant that there was a problem with urlEncode and
>> urlDecode as per bug
>> https://quality.livecode.com/show_bug.cgi?id=14015 that your
>> references Monte. However, I now consider that urlEncode and urlDecode
>> are NOT broken and bug 14015 is really a Documentation bug that the
>> urlEncode and urlDecode Dictionary entries should be updated.
>
>> Also, the detailed files and detailed folders doesn't need an array
>> returned (although that would be nice). This issue is the
>> percent-encoding is not following the standard convention of UTF8
>> encoding non-ASCII characters before percent encoding. LC should
>> follow industry conventions in this regard.
>
> Strictly speaking urlEncode and urlDecode are indeed not broken - they 
> are just very old. They were implemented when there were no 
> universally accepted industry conventions for encoding of URLs 
> (particularly those containing international characters).
>
> They cannot be changed for obvious reasons - but we could do with 
> urlEncodeRFC and urlDecodeRFC which would be string->string functions 
> which do what is expected today. (And also not encode some characters 
> in a way which is now not the 'standard du jour').
>
> For the same reason that urlEncode and urlDecode cannot be changed, 
> the detailed files/folders cannot be changed.
>
> My general feeling here is that 'the detailed files' and 'the detailed 
> folders' should be put out to pasture as they are grossly inefficient 
> and difficult to work with.
>
> Instead new functions 'fileInfo' and 'folderInfo' could replace them. 
> They would take one or two parameters, the first a filename and the 
> second an optional attribute.
>
> If no attribute is specified then an array of all known attributes for 
> the platform is returned; otherwise that specific attribute is returned.
>
> e.g. fileInfo("foo.txt", "modification time") -> returns the 
> modification time of foo.txt
>      fileInfo("foo.txt") -> returns an array with keys such as 
> "modification time" mapping to the obvious thing
>
> I'd probably vote for the functions returning empty if the target 
> didn't exist (rather than throwing an error), for the simple reason as 
> you can't check for existence of a file and get its details in an 
> atomic fashion (another process could decide to delete the file you 
> just asserted existed between the 'there exists' and the 'fileInfo' 
> call).
>
> Warmest Regards,
>
> Mark.
>