MD5 digests of very big files

[Alejandro Tejada]

Hi Malte,

Malte Pfaff Brill wrote:
> I guess the problem you see is that revs MD5s
> need to be binaryDecoded to have the same output
> like for example PHP. A simple script to see this is:
> on mouseUp
>   local tCheck,tResult
>   put md5Digest("moo") into tCheck
>   get binaryDecode("H*",tCheck,tResult)
>   put tCheck&cr&tResult
>  -- tCheck is revs, tResult is compliant to PHP
> end mouseUp

Excellent! :-D
Now the quasiMD5 function reads as follow:

function quasiMD5 pFile 
-- posted by Mark Waddingham 
  local tMD5s,tResult
  open file pFile for binary read 
  repeat 
    read from file pFile for 4096 bytes 
    if it is empty then 
      exit repeat 
    end if 
    put the md5digest of it after tMD5s 
  end repeat 
  close file pFile 
  put the md5Digest of tMD5s into tCheck
  get binaryDecode("H*",tCheck,tResult)
  return tResult
end quasiMD5 

Just notice that converting the MD5 binary string
to Hexadecimal numbers do not solve the last
question:

How could we change this  function to get the same MD5
from a big file (like a Linux ISO image) than command
lines programs do?

For example, the MD5 of the more recent version of Puppy
Linux ISO image is 0847331e9cdcb55eaae238924d131a24
ftp://ibiblio.org/pub/linux/distributions/puppylinux/puppy-5.0.1/

This quasiMD5 function does not produce the same MD5
digest and everyone advice that we should not load the
full 128 MB file on memory to calculate it using Rev.

I am sure that command line programs that calculate MD5
digest only read small chunks too (just like this function
does) but they also apply other math or bit operations to
get the correct MD5 digest.

My question is: Which other operations are needed in
this quasiMD5 function to produce true MD5 digests
of big files?

Thanks in advance.

Al
-- 
View this message in context: http://runtime-revolution.278305.n4.nabble.com/MD5-digests-of-very-big-files-tp2271355p2272645.html
Sent from the Revolution - User mailing list archive at Nabble.com.