Deleting Data Woefully Slow

[Mark Wieder]

Kay-

Thursday, March 25, 2010, 3:20:41 AM, you wrote:

> If anyone thinks they can delete data faster than my Test 4 I'd sure
> appreciate some pointers :-)

Here's what I threw together, and for smaller data sets it's roughly
twice as fast as your test 4 (1 or 2 ms vs 4 or 5 ms over 5000 lines).
But for larger data sets it starts to bog down and then comes out
twice as slow (90 ms vs 45 ms over 50000 lines). And since I'm working
with a reduced data set after the filter command, using the "delete"
line instead of the "put" line makes no difference to the overall
timing. If you know in advance that you want to delete three lines
starting with a trigger phrase, then you've got a fast algorithm
there.

    --test5
    put the millisec into tStart
    put tData into tData5
    filter tData5 with "*a*"
    repeat for each line tLine in tData5
        put lineoffset(tLine, tData) into tPos
        if tPos is not 0 then
            --delete line tPos to tPos+2 of tData
            put line 1 to tPos-1 of tData & cr & line tPos+3 to -1 of tData & cr into tData
        end if
    end repeat
    put word 1 to -1 of tData into tData
    put the millisec into tEnd
    put tEnd - tStart into tTotal5
    
    if (tData1 <> tData) then
        answer "Error: tData1 <> tData"
        breakpoint
    end if

-- 
-Mark Wieder
 mwieder at ahsoftware.net