Speed up a slow loop

[William Prothero]

Folks:
I've been following this thread in wonder at the creative ways of optimizing this word search. I hope that the code will be published somewhere, available to all, with credits to the multiple idea authors.

Kudos and congratulations to those who offered such creative and thoughtful ideas. I'm awed!

Bill

William A. Prothero, PhD
Prof Emeritus, Dept of Earth Science
University of California, Santa Barbara

> On Mar 6, 2022, at 5:33 AM, Alex Tweedly via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> 
>> On 06/03/2022 06:35, J. Landman Gay via use-livecode wrote:
>> 
>> Combined with Ken's suggestion to use "difference", looking up a list of about 50 words in the filtered dictionary reduces the time from 2-4 seconds to about 40 ms. It varies, depending on a lot of things, but it's fast enough that you don't notice. That's on my Pixel 5; on the Mac a lookup takes 5ms. And I can do it all in LC script.
>> 
> 5ms !?!
> 
> I did a (very simple) test  (see code below)
> 
>  - take the whole sowpods.txt file (267k words)
> 
>  - create a set (i.e. split by CR as set)
> 
>  - look up 50 randomly chosen words + 5 non-words (just in case failed searches were expensive).
> 
> Doing this 100 times takes 6-10 ms (on an old MacBook Pro).
> 
> Could you maybe post the code that you're using that takes 5ms ?
> 
>> I love these little speed contests we have here. Thank you.
>> 
> Alex.
> 
>> 
>> on mouseup
>>    local tmp, tNumberOfLines, tNumberOfLoops
>>    local tWords, tAWords, tTries
>> 
>>    put the cWords of me into tWords
>>    if tWords is empty then
>>       put URL ("file:" & specialfolderpath("resources") & "/sowpods.txt") into tWords
>>       set the cwords of me to tWords
>>       put "From file" &CR into fld "F"
>>    else
>>       put "using custom property" &CR into fld "F"
>>    end if
>> 
>>    put tWords into tAWords
>>    split tAWords by CR as set
>> 
>>    repeat with I = 1 to 50
>>       put random(the number of lines in tWords) into tmp
>>       put line tmp of tWords &CR after tTries
>>    end repeat
>>    repeat with I = 1 to 5
>>       put "azazaz" &Cr after tTries
>>    end repeat
>> 
>>    put 100 into tNumberofLoops
>> 
>>    local t1, t2, tCount, tTotal
>> 
>>    --   put the millisecs into t1
>>    --   repeat tNumberOfLoops times
>>    --      put 0 into tCount
>>    --      repeat for each line L in tTries
>>    --         if L is among the lines of tWords then add 1 to tCount
>>    --      end repeat
>>    --   end repeat
>>    --   put the millisecs into t2
>>    --   put tCOunt && "iterate lines" && t2-t1 &CR after fld "F"
>> 
>>    put the millisecs into t1
>>    repeat tNumberOfLoops times
>>       put 0 into tCount
>>       put 0 into tTotal
>>       repeat for each line L in tTries
>>          add 1 to tTotal
>>          if tAWords[L] then add 1 to tCount
>>       end repeat
>>    end repeat
>>    put the millisecs into t2
>>    put tCount && "of" && tTotal && "each element" && t2-t1 &CR after fld "F"
>> 
>> 
>> end mouseup
> 
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode