Speed testing: Fastest search method

Mike Bonner bonnmike at gmail.com
Sun Aug 31 11:24:20 EDT 2014


Heres the code i'm testing with.




on mouseUp
--## First Method, delete line
   put field "dat" into tDat
   put the number of lines in tDat into tLines
   put the millisec into tStart
   put 1 into x
   repeat for each line tLIne in tDat
      if item 1 of tLine < 40 then
         put x & comma after tDeletions
      end if
      add 1 to x
   end repeat
   delete the last char of tDeletions
   sort items of tDeletions descending numeric -- reverse so the delete
will work correctly

   repeat for each item tItem in tDeletions
      delete line tItem of tDat --delete the lines after the repeat for
each.
   end repeat

   put tDat into field "outField"


   put merge("It took [[the millisec - tStart]] Milliseconds") after msg
   put cr & merge("There were [[tLInes]] Lines to start, and [[the number
of lines in tDat]] lines after elimination") after msg

-- ####next method
put field "dat" into tDat
   put the millisec into tStart
   put 1 into x
   repeat for each line tLIne in tDat
      if item 1 of tLIne < 40 then
         put "" into line x of tDat
      end if
      add 1 to x
   end repeat
   filter tDat without empty
   put tDat into field "outField"
   put cr & merge("It took [[the millisec - tStart]] Milliseconds") after
msg
   put cr & merge("There were [[tLInes]] Lines to start, and [[the number
of lines in tDat]] lines after elimination") after msg

end mouseUp

And here is the code to gen data.

on mouseUp
   repeat 80000 times --
      put random(500) & comma & any item of
("mike,tom,joe,mary,tracy,janice") & comma & random(400) & cr after tDat
   end repeat
   delete the last char of tDat
   put tDat into field "dat"
end mouseUp

On windows 7, lc 6.6.2, method 1 is nearly twice as fast with data that
removes 6k+ lines. In addition, after comparing data, the filter method
isn't working correctly, most likely due to working with tDat during the
for each. ?  Did someone mention that this has been changed? If so, what
version do I need to be running to make the filter method work correctly?

The problem with the delete method of course, is that the more lines are
deleted, the more speed penalty (also, if they're weighted towards the end
of the data)  Seems though, that hitting tDat using line numbers, whether
deleting, or changing, would have similar speed penalties.  Also, if the
criteria is not too complicated (say, all items that are less than 40 as in
my example)  perhaps a pre sort, search for the first matching line, locate
the last matching line, then delete lines ....... all in one hit.


On Sun, Aug 31, 2014 at 8:49 AM, Mike Bonner <bonnmike at gmail.com> wrote:

> Method 2 doesn't have the filter, I am guessing the lack is a typo?  Also,
> I'd be interested in your results with the repeat for each method that
> modifies tvar with the direct line deletion, though thinking about it,
> (since i'm awake now) it wouldn't work unless it was modified.  Something
> like..
>
> on mouseUp
>    set the cursor to watch
>    put the long seconds into tStart
>    put fld "data" into tVar
>    repeat for each line L in tVar
>       add 1 to x
>       if L="" then
>          delete line x of tVar
>          subtract 1 from x -- to allow for the deleted line. Also still
> wonder about modifying tvar directly in the repeat for each.
>
>       end if
>    end repeat
>    put tVar into fld "output"
>    put the long seconds - tStart into fld "timer2"
> end mouseUp
>
>
>
> On Sun, Aug 31, 2014 at 8:21 AM, <dfepstein at comcast.net> wrote:
>
>> <admin at FlexibleLearning.com> compared 3 methods of stripping lines from
>> a variable, and concluded:
>> If memory is an issue, then Method 2 is best
>> If memory is not an issue, then Method 3 is best
>>
>> 3 questions:
>>
>> 1. Is there a good way to determine ahead of time whether memory is an
>> issue? When I start the handler I can find out how big tVar is, but how do
>> I find out how much memory is available?
>>
>> 2. Does this step in all 3 handlers --
>> put fld "Data" into tVar
>> -- itself use up memory? If fld "Data" is occupying a gigabyte of RAM,
>> does writing it to tVar use another gigabyte?
>>
>> 3. Method #2 appears to me to violate the rule against modifying the
>> variable to which you are applying "repeat for each":
>> #2:
>> on mouseUp
>> set the cursor to watch
>> put the long seconds into tStart
>> put fld "data" into tVar
>> repeat for each line L in tVar
>> add 1 to x
>> if L="" then
>> put "" into line x of tVar -- RIGHT HERE, WE'RE MODIFYING tVar!
>> end if
>> end repeat
>> put tVar into fld "output"
>> put the long seconds - tStart into fld "timer2"
>> end mouseUp
>>
>> Have I misunderstood that rule?
>>
>> Many thanks.
>>
>> David Epstein
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>
>



More information about the use-livecode mailing list