use-livecode Digest, Vol 224, Issue 8

[Bob Sneidar]

You can read for x characters then check the result for empty. So no you do not need to read the entire file into memory. 

Bob S


> On May 10, 2022, at 12:14 , Mark Clark via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Thanks Tom, Mike and Craig. Sorry for the late response, I’m on the digest version.
> 
> My thought was that an open file for read does not require placing the whole file into memory. Am I mistaken in that assumption? Seems like it would be nice to not use all of the available memory to run the decrypt. Results below seem to confirm this...
> 
> I worked up a quick stack using the chunking notion I referenced in the original post and this seems to work fine. I compressed the Mac system folder on an old SSD. That directory was 8.9GB on disk. Compressing it with the Finder took about 28 minutes. Compressed size was 5.9GB.
> 
> Using a byte range of 1048576 (1MB—small but useful for testing), I was able to read the source compressed file, encrypt and write to the new (encrypted) file in about 2 minutes. Decrypt took a little longer (less than 3 min). Memory utilization (I did a quick build of the stack) was 44-45MB during both operations. I’ll bump up the byte range and see how that compares. 
> 
> 
> 
> Thanks,
> Mark
> 
> 
>> On May 10, 2022, at 11:00 AM, use-livecode-request at lists.runrev.com <mailto:use-livecode-request at lists.runrev.com> wrote:
>> 
>> there is no way to decrypt something that does not fit in memory. with 64
>> bit builds the limit is whatever the motherboard  supports.   on 32 bit
>> builds the limit is whatever the os will allow 1 process to have,
>> but then u need memory to store the decrypted data too.
>> you can use a command line program to outsource that work and memory
>> management.
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode