put url some url into someVar: nasty crash :(

[Mark Waddingham]

Hi David,

Mark Schonewille was trying to do something very similar a while ago and
filed an enhancement request about being able to do md5 digests on large
files *without* them needing to be loaded. For interest see here:
 
  <http://support.runrev.com/bugdatabase/show_bug.cgi?id=2410>

As suggested (by me) in the bug-report, you needn't take the md5digest
of the entire file at once and can do something like this instead:

function quasiMD5 pFile
  local tMD5s
  open file pFile for binary read
  repeat 
    read from file pFile for CHUNK_SIZE chars
    if the result is EOF then
      exit repeat
    end if
    put the md5digest of it after tMD5s
  end repeat
  close file pFile
  return the md5Digest of tMD5s
end quasiMD5

Where you can make CHUNK_SIZE some suitable size (perhaps 256k/512k).

[ My intuitive analysis of the impact of doing this on the integrity
(i.e. potential for collision) of the digest is that it will be minimal
- but perhaps someone more knowledgeable in this area could comment. ]

Hope this helps,

Mark.

On Sun, 2005-09-25 at 21:50 +0200, david bovill wrote:
> My script keeps crashing Rev... and I think it is a simple as this line:
> 
>       put url ("file:" & someFile) into someData
> 
> But if the file is big enough - in my case a 900MB video file (or  
> anything larger) - instant crash. No graceful wait for virtual memory  
> to kick in :( This is OSX with 1GB memory - not OS9! Can this be right?
> 
> Can't think of any other way of doing what i need - using the md5Hash  
> function to uniquely identify the video files and associated text  
> files. Is this a problem on Linux and Windows?
> 


------------------------------------------------------------------
 Mark Waddingham ~ 36degrees at runrev.com ~ http://www.runrev.com
       Runtime Revolution ~ User-Centric Development Tools