Unicode is not "everywhere"...

[Monte Goulding]

> On 23 Aug 2019, at 11:07 am, Paul Dupuis via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> I reported what I thought was 3 bugs in 1 report in https://quality.livecode.com/show_bug.cgi?id=22213 <https://quality.livecode.com/show_bug.cgi?id=22213>. I have edited that report to focus on a single bug - that the detailed files (and probably the detailed folders) is broken for Unicode as every Unicode character in a file name is encoded as %3F or ?. Originally I had though that meant that there was a problem with urlEncode and urlDecode as per bug https://quality.livecode.com/show_bug.cgi?id=14015 <https://quality.livecode.com/show_bug.cgi?id=14015> that your references Monte. However, I now consider that urlEncode and urlDecode are NOT broken and bug 14015 is really a Documentation bug that the urlEncode and urlDecode Dictionary entries should be updated.
> 
> If you look at the Wikipedia entry for URL (or percent) encoding, it states that the standard practice for non-ASCII character in a URL is to encode them as UTF8 BEFORE percent-encoding. There fore, urlDecode and urlEncode are working correctly by the accepted standards. The Dictionary entries need updating to note that any non-ASCII text should be UTF8 encode before urlEncoding and UTF8 decoded after urlDecoding. See my DOC bugs:
> 
> https://quality.livecode.com/show_bug.cgi?id=22340 <https://quality.livecode.com/show_bug.cgi?id=22340>
> and
> https://quality.livecode.com/show_bug.cgi?id=22341 <https://quality.livecode.com/show_bug.cgi?id=22341>
> 
> I believe these replace bug https://quality.livecode.com/show_bug.cgi?id=14015 <https://quality.livecode.com/show_bug.cgi?id=14015>

Hmm… ok that’s a fair point. We really do need a standards compliant urlEncode function though which could resolve this because if it gets a string parameter it will know it should convert to UTF8. Again it would need a parameter or something.
> 
> Also, the detailed files and detailed folders doesn't need an array returned (although that would be nice). This issue is the percent-encoding is not following the standard convention of UTF8 encoding non-ASCII characters before percent encoding. LC should follow industry conventions in this regard.

Yes, although I think we can’t do that in a backwards compatible way which is why I suggested an array option.

Cheers

Monte