LC 9 and Memory

[Mark Waddingham]

On 2018-08-13 20:01, Stephen MacLean via use-livecode wrote:
> Hi Mark,
> 
> Yes, doing quite a bit, which LC handles just fine:) Except for these
> with big queries;) I do want to mention that I call plenty of
> functions, so it’s not a real tight loop, just a loop with a lot going
> on.
> 
> Ok, will do the next time I run one of these… These big batches take
> about 5 hours to run.

 From what you said in your other message this could well be a 
memory-leak...

However, the other potential cause is what's called 
'heap-fragmentation'. Whilst memory is being freed correctly, sometimes 
certain use patterns can cause the C heap to consist of 'mostly free 
space', but having no contiguous block large enough to allocate some of 
the blocks of memory you are using - so the heap gets extended. When you 
do free a 'big block' it gets used up by lots of small blocks.

Basically, the heap becomes like a block of swiss-cheese - you can fit 
small things in the holes, but nothing big.

The 'heap' report will help determine what's going on there (you don't 
need to run it until you get to the exhaustion point - just for a while 
until you see the gradually continual rise).

By looking at the report it is possible to tell the actual % of memory 
in use in the heap overall.

Is there any way to split up the processing of a single batch into 
pieces, each run in its own process (sequentially?) - that approach 
actually gives you two things - (1) a fix to the exhaustion problem you 
are having and (2) the ability to recover most of the batch if one piece 
fails.

Warmest Regards,

Mark.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps