LC 9 and Memory

[Stephen MacLean]

Hi Mark,

Here is one from when the app hit the half way point in it’s processing run:

Process:         LiveCode-Business [799]
Path:            /Applications/LiveCode Business 9.0.1 (rc 2).app/Contents/MacOS/LiveCode-Business
Load Address:    0x10d387000
Identifier:      com.runrev.livecode
Version:         9.0.1.15101 [RC 2] (9.0.1.15101 [RC 2])
Code Type:       X86-64
Parent Process:  ??? [1]

Date/Time:       2018-08-26 01:24:06.956 -0400
Launch Time:     2018-08-25 22:42:23.095 -0400
OS Version:      Mac OS X 10.12.6 (16G1510)
Report Version:  7
Analysis Tool:   /Applications/Xcode.app/Contents/Developer/usr/bin/heap
Analysis Tool Version:  Xcode 9.2 (9C40b)
----

Process 799: 6 zones
Zone DefaultMallocZone_0x10ee3a000: Overall size: 149641216KB; 520214585 nodes malloced for 28100061KB (18% of capacity); largest unused: [0x6003c9060040-1664KB]
Zone DefaultPurgeableMallocZone_0x11264e000: Overall size: 49836KB; 75 nodes malloced for 49828KB (99% of capacity); largest unused: [0x11daad000-11352KB]
Zone GFXMallocZone_0x10e5d5000: Overall size: 10240KB; 46 nodes malloced for 8KB (0% of capacity); largest unused: [0x7fd853001400-8187KB]
Zone MallocHelperZone_0x10e4f0000: Overall size: 2976716KB; 508527 nodes malloced for 1998382KB (67% of capacity); largest unused: [0x119aa3000-12876KB]
Zone QuartzCore_0x111cac000: Overall size: 20480KB; 4 nodes malloced for 5KB (0% of capacity); largest unused: [0x7fd851800000-8192KB]
Zone x-alloc_0x7fd84b028000: Overall size: 20KB; 5 nodes malloced for 1KB (5% of capacity); largest unused: [0x111cde000-8KB]
All zones: 520723242 nodes malloced - 30148282KB

I’d love to send you that one from near the end… LC was taking up almost 59GB of memory, memory pressure was insane, over 30 GB of compressed memory, 15GB swap file, finder having me force quit the few apps I had open. Tried to run heap, but after 3 hours it caused a kernel panic and the computer rebooted.


I can send you the full file if needed. Any help is appreciated.

Thanks,

Steve MacLean

> On Aug 13, 2018, at 2:07 PM, Mark Waddingham via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> On 2018-08-13 20:01, Stephen MacLean via use-livecode wrote:
>> Hi Mark,
>> Yes, doing quite a bit, which LC handles just fine:) Except for these
>> with big queries;) I do want to mention that I call plenty of
>> functions, so it’s not a real tight loop, just a loop with a lot going
>> on.
>> Ok, will do the next time I run one of these… These big batches take
>> about 5 hours to run.
> 
> From what you said in your other message this could well be a memory-leak...
> 
> However, the other potential cause is what's called 'heap-fragmentation'. Whilst memory is being freed correctly, sometimes certain use patterns can cause the C heap to consist of 'mostly free space', but having no contiguous block large enough to allocate some of the blocks of memory you are using - so the heap gets extended. When you do free a 'big block' it gets used up by lots of small blocks.
> 
> Basically, the heap becomes like a block of swiss-cheese - you can fit small things in the holes, but nothing big.
> 
> The 'heap' report will help determine what's going on there (you don't need to run it until you get to the exhaustion point - just for a while until you see the gradually continual rise).
> 
> By looking at the report it is possible to tell the actual % of memory in use in the heap overall.
> 
> Is there any way to split up the processing of a single batch into pieces, each run in its own process (sequentially?) - that approach actually gives you two things - (1) a fix to the exhaustion problem you are having and (2) the ability to recover most of the batch if one piece fails.
> 
> Warmest Regards,
> 
> Mark.
> 
> -- 
> Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode