How to filter a big list

[Jérôme Rosat]

Jim,

Le 22 oct. 2009 à 05:41, Jim Ault a écrit :
> your asking a lot of a chunking function to scan a large body of  
> text between key strokes.
> Start with the following steps to see if these help.
> -1-  Showing a list of more than 50 hits may not be useful
Good idea.
> -2-  Doing an filter operation with less than 3 chars may not be  
> useful
Filter a list with less than 3 chars reduce the list.
> -3-  Showing the number of lines (hits) at the top of the field is  
> useful
Good idea.
> -4-  Most likely you will need to pre-index the 400K lines to get  
> more speed
> Indexing is what data bases do to boost speed.  You need to decide  
> what the logic is, such as any char in any string, or words  
> beginning with the user input, etc.
I should think to this solution.
> Is the 400K set of lines dynamic or static?
Static for my tests. Dynamic from 0 to about 400k lines in production.
> Does the user type logical words, or phrases?
Both.
> eg.  santos  -- single word
> eg.  Gourgas  -- single word
> eg.  dos santos  -- phrase in order
> eg.  rue Gourgas  --phrase in order
> If link tables are required, then you should consider a database,  
> since this is something they do well.
>
> Hope this helps.

Yes, thank you.

If some body in the list is looking for a file with names and address  
(400K lines), I create a file with french name, French, Spanish,  
Italians, Portuguese first name and address of Geneva in Switzerland.  
It is possible to download here: http://files.me.com/jrosat/tac1b4

In the state of my tests, here the my “fastest” code:

on keyUp
    set the itemdelimiter to tab
    Switch the number of chars of me
       Case 1
          put empty into vListe
          repeat for each line theLine in vNoms
             if item 2 of theLine contains me then put theLine & cr  
after vListe
          end repeat
          break
       Case 2
          put vListe into maListe
          put empty into vListe
          repeat for each line theLine in maListe
             if item 2 of theLine contains me then put theLine & cr  
after vListe
          end repeat
          break
       Default
          put vListe into maListe
          put empty into vListe
          repeat for each line theLine in maListe
             if item 2 of theLine contains me then put theLine&cr  
after vListe
          end repeat
          delete the last char of vListe
          put "Number of lines =" && (the number of lines of vListe)  
into theOutput
         if the number of lines in vListe > 60 then
             put theOutput & cr & line 1 to 60 of vListe & cr & "..."  
into field "fListe"
          else
             put theOutput & cr & vListe into field "fListe"
          end if
    end Switch
end keyUp

Jerome Rosat