use-livecode Digest, Vol 224, Issue 8
bobsneidar at iotecdigital.com
Tue May 10 18:30:46 EDT 2022
You can read for x characters then check the result for empty. So no you do not need to read the entire file into memory.
> On May 10, 2022, at 12:14 , Mark Clark via use-livecode <use-livecode at lists.runrev.com> wrote:
> Thanks Tom, Mike and Craig. Sorry for the late response, I’m on the digest version.
> My thought was that an open file for read does not require placing the whole file into memory. Am I mistaken in that assumption? Seems like it would be nice to not use all of the available memory to run the decrypt. Results below seem to confirm this...
> I worked up a quick stack using the chunking notion I referenced in the original post and this seems to work fine. I compressed the Mac system folder on an old SSD. That directory was 8.9GB on disk. Compressing it with the Finder took about 28 minutes. Compressed size was 5.9GB.
> Using a byte range of 1048576 (1MB—small but useful for testing), I was able to read the source compressed file, encrypt and write to the new (encrypted) file in about 2 minutes. Decrypt took a little longer (less than 3 min). Memory utilization (I did a quick build of the stack) was 44-45MB during both operations. I’ll bump up the byte range and see how that compares.
>> On May 10, 2022, at 11:00 AM, use-livecode-request at lists.runrev.com <mailto:use-livecode-request at lists.runrev.com> wrote:
>> there is no way to decrypt something that does not fit in memory. with 64
>> bit builds the limit is whatever the motherboard supports. on 32 bit
>> builds the limit is whatever the os will allow 1 process to have,
>> but then u need memory to store the decrypted data too.
>> you can use a command line program to outsource that work and memory
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
More information about the use-livecode