RB clear winner in speed test

[Brian Yennie]

On guard! Another Rev entry!

Notes:

1) I went back to building the mini-index of first letter hints from 
words, and caching the index for future searches
2) I took advantage of the caseSensitive property, which noticeably 
speeds up offset()
2a) I noted that the position in the full text has to be at least 2x-1 
that of the 1 character index
3) I used a quick array to keep track of two-word combos and quickly 
eliminate any line which has a non-existent one in it
4) I used lock/unlock screen because the screen was previously being 
updated 6 times instead of once (once for each put into a field, 
including the timing mechanism).

My benchmarks are:

HARDWARE: iBook 500Mhz - has been benching at pretty close to 50% speed 
of Chris' test machine
FIRST RUN (which creates cached index): 130-135 ticks
ADDITIONAL RUNS (with cached index): 58-62 ticks

Cut those in half, and we would have 65-68 ticks on the first round and 
29-31 on each additional, which would actually be faster than the RB 
bench!

Of course only Chris can tell us how they run on his end =).


##########

local shortFindList, isWord, textDigest, tTime

on mouseUp
   local tMatchList, sMS, fn, stList, shortFind, w, w0, c

   ## if you want to force refresh of index
   if (the optionKey is "down") then put empty into textDigest

   set cursor to watch

   put the milliseconds into sMS

   lock screen
   put empty into tTime

   ## use URL syntax for faster file loading
   put "./TargetText.txt" into fn
   put url ("file:"&fn) into tMatchList

   put "./SearchTextList.txt" into fn
   put url ("file:"&fn) into stList

   put timeit(sMS,"Load Data") into sMS
   -- stripping quotes from both sets saves problems with single quotes
   --  or with quoted strings being treated as a word

   replace quote with empty in stList
   put lower(tMatchList) into tLowerMatchList
   replace quote with empty in tLowerMatchList

   put md5Digest(tMatchList) into tDigest
   if (tDigest <> textDigest) then
     put empty into shortFindList
     repeat for each word w in tLowerMatchList
       put char 1 of w into c
       put c after shortFindList
       put TRUE into isWord[w]
       put TRUE into isWord[w0&w]
       put w into w0
     end repeat
     put tDigest into textDigest
   end if

   set the caseSensitive to TRUE

   repeat for each line inLine in stList
     put empty into shortFind
     put lower(inLine) into inLineLower
     put 0 into i
     put empty into w0
     repeat for each word w in inLineLower
       if not isWord[w0&w] then
         put empty into shortFind
         exit repeat
       end if
       put char 1 of w after shortFind
       put w into w0
     end repeat
     if (shortFind is empty) then next repeat
     put offset(shortFind, shortFindList) into x
     if (x > 0) then
       if (offset(inLineLower, tLowerMatchList, x*2-1) > 0) then
         put inLine & return after MatchList
       end if
     end if
   end repeat
   put timeit(sMS, "Check Matches") into sMS
   put MatchList into fld "TheResults"
   put stList into fld "SearchTextList"
   put tMatchList into fld "TargetText"
   put timeit(sMS,"Display All") into sMS
   put tTime&cr after fld "SpeedRecords"
   unlock screen
end mouseUp