Accumulating text is *VERY* slow in LC9 on Windows

[Mark Waddingham]

On 2021-08-25 18:15, Ben Rubinstein via use-livecode wrote:
> I simplified it down to this (pointless) loop which just rebuilds a
> table one line at a time:
> 
>    local tNewTable
>    repeat for each line tRow in tWorkTable
>       put tRow & return after tNewTable
>    end repeat
> 
> with these results:
> 
> 	6.7.11 MacOS	  8 seconds
> 	6.7.11 Win32	  7 seconds
> 	9.6.3  MacOS	  0 seconds
> 	9.6.3  Win32	591 seconds

Using a buffer var should workaround the performance issue (which is 
related to the windows heap manager not being very good at continually 
re-extending a buffer):

on mouseUp
    local tLine
    repeat 256 times
       put "*" after tLine
    end repeat

    local tTime
    put the millisecs into tTime

    local tBuffer

    local tText
    repeat 257000 times
       put tLine & return after tBuffer
       if the number of codeunits in tBuffer > 500000 then
          put tBuffer after tText
          delete codeunit 1 to -1 of tBuffer
       end if
    end repeat
    put tBuffer after tText
    answer (the number of codeunits in tText) & return & (the millisecs - 
tTime)
end mouseUp

In the original loop, tNewTable is continually extended internally, 
something which appears to cause O(n^2) performance on Windows.

In the revised loop, an intermediate buffer var is used which (after 
first time getting 'full') will have a backing store of 500k ish - 
meaning tText is extended much less often. (Playing with the value of 
500000 up or down will affect the resulting speed - there will always be 
a sweet spot).

On my Windows VM - the above loop (which generates about 68mb of text or 
so, takes about 3s.

Hope this helps!

Mark.

P.S. This is an engine issue - we'll need to look into why there's such 
a difference with 6.7 - as, I'm pretty sure I kept the rules about 
extending buffers pretty much the same in string concatenation.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps