"the files", "get URL" and Unicode File Names

[Igor de Oliveira Couto]

Dear List Members,

I am trying to write a reasonably simple test script, which would iterate through every file in a chosen directory, and give me the md5 checksum for that file. I am coming across a problem, which *may* be related to Unicode, but I'm not certain. The algorithm is like this:

* user selects directory
* get list of files in directory (using 'the files')
* for every file in the list:
** get the file data stream (using 'get URL "binfile:..."')
** get the checksum

The function that gets the datastream using 'get URL' returns 'empty' in case it cannot find the file. I am finding, that when a file has certain accented or foreign characters in its name, 'get URL' is unable to find it - even though it is using the original unmodified string returned by 'the files'... What is most puzzling, is that this does not happen with *every* file that has an international character in it - files with french, spanish, german or even scandinavian characters fly through without a hitch. But if I have a file that has a "ĉ" (c+circumflex) or "ŭ" (u+breve) in its name, for instance, it chokes.

Am I doing something wrong, or missing something basic? - or did I hit a limitation, or bug?

Many thanks for any guidance,

--
Igor de Oliveira Couto
Sydney, Australia


PS - the code in full, for those interested:

1) Make a new Mainstack, and add a field named "folderContents", and a button.
2) Put the following into the button's script:

on mouseUp
   answer folder "Please select a folder:"
   if it is empty then exit mouseUp
   local tDefault, tItems
   put the defaultFolder into tDefault
   set the defaultFolder to it
   put empty into field "folderContents"
   put the files into tItems
   repeat for each line xLine in tItems
      put "name=" & quote & xLine & quote after field "folderContents"
      put " checksum=" & quote & fileDigest(the defaultFolder & "/" & xLine) & quote & return after field "folderContents"
   end repeat
   filter field "folderContents" without empty
   set the defaultFolder to tDefault
end mouseUp

function hexDigest pvalue 
   local tRes, tMD5
   put md5Digest(pValue) into tMD5 
   get binaryDecode("H*",tMD5,tRes) 
   return tRes 
end hexDigest

function fileDigest pFile
   if there is a file pFile then
      get URL ("binfile:" & pFile)
      return hexDigest(it)
   else
      return empty
   end if
end fileDigest