use-livecode Digest, Vol 224, Issue 8

[Mark Clark]

Thanks Tom, Mike and Craig. Sorry for the late response, I’m on the digest version.

My thought was that an open file for read does not require placing the whole file into memory. Am I mistaken in that assumption? Seems like it would be nice to not use all of the available memory to run the decrypt. Results below seem to confirm this...

I worked up a quick stack using the chunking notion I referenced in the original post and this seems to work fine. I compressed the Mac system folder on an old SSD. That directory was 8.9GB on disk. Compressing it with the Finder took about 28 minutes. Compressed size was 5.9GB.

Using a byte range of 1048576 (1MB—small but useful for testing), I was able to read the source compressed file, encrypt and write to the new (encrypted) file in about 2 minutes. Decrypt took a little longer (less than 3 min). Memory utilization (I did a quick build of the stack) was 44-45MB during both operations. I’ll bump up the byte range and see how that compares. 



Thanks,
Mark


> On May 10, 2022, at 11:00 AM, use-livecode-request at lists.runrev.com <mailto:use-livecode-request at lists.runrev.com> wrote:
> 
> there is no way to decrypt something that does not fit in memory. with 64
> bit builds the limit is whatever the motherboard  supports.   on 32 bit
> builds the limit is whatever the os will allow 1 process to have,
> but then u need memory to store the decrypted data too.
> you can use a command line program to outsource that work and memory
> management.