Speed up a slow loop

[Neville Smythe]

I just ran a test to try to find out how much RAM is used when you create a large array; the result is rather surprising.

 BTW Bob: I did not imply that an array is implemented in LC as a database. Merely that to work its magic it must be indexing keys somehow. 

I had suggested a 2.5 MB list of 275K words might expand to no worse than 10MB when split into an array. Not quite, as it turns out.

I wrote a very simple stack which just has two buttons. 

The first reads SOWPODS.txt into a global variable gWordlist. Don’t try putting that variable into the msg box — LC becomes unusable with the Beachball of Hope spinning for a few seconds whenever you do anything. Aha, it is the LC engine that is having problems with large data (in this case displayed in a field of course, not in the variable itself)

The second button puts gWordlist into aa, and splits aa by CR  as a set.

I then turned the stack into a standalone and looked at the memory used in the Mac Console

. When the stack launches, real memory used is 29.8 MB (in a second run 45 MB)
        That’s the basic LC engine + 18 MB of Mac OS stuff which is shared with all apps, the stack itself would be tiny

. after button 1, reading in the 2.5 MB file, the real memory used is 46 MB (second run 56 MB)
        So 16 MB to store a 2.5 MB file in a variable (in the second run 11 MB, I guess 5 MB of engine code was garbage collected)

. After button 2, the real memory used is 121 MB (second run 131 MB)
        So splitting a 2.5 MB file into an array with 275 K keys required 75 MB (this figure was duplicated in the second run)

We are now talking pretty large amounts of memory that the LC engine has to move around, for example probably making copies in its procedure stacks (that's stack in the coding sense).

I'm not surprised that LC might be getting indigestion with variables of this size on a small device. I seem to remember running into RAM allocations for an iOS app measured in the low hundreds of MB.

Neviille