How to filter a big list

[Jérôme Rosat]

Thank you Jim, Richard, Brian and Mark,

Please excuse me to answer so tardily, I posted a message yesterday,  
but it was not published in the list. I make a new attempt.

I explained in my message that I wish to filter a list of names and  
addresses dynamically when I type a name in a field. This list  
contains 400'000 lines like this:  Mme [TAB] DOS SANTOS albertina  
[TAB] rue GOURGAS 23BIS [TAB] 1205 Genève

I made various tests using the "repeat for each" loop and the  
"filter ... with" command. Filtering takes the most time when I type  
the first and the second letter. That takes approximately 800  
milliseconds for the first char and about 570 milliseconds for the  
second char. The repeat loop with the "contains" operator is a little  
beat slower (about 50 milliseconds) than the "filter ... with". There  
is no significant difference when the third char or more is typed. Of  
course I filter a variable before to put it in the list field.

Obviously, 800 milliseconds to filter a list of 400'000 lines, it is  
fast. But it is too slow for what I want to do. It would take a time  
of filtering lower than 300 milliseconds so that the user is not  
slowed down in his typing.

Sorry to have been insufficiently precise in my first message. I  
continue my tests and I will publish the fastest code.

Jerome Rosat

Le 20 oct. 2009 à 03:41, Jim Ault a écrit :
>
> First, what do you mean by 'slow' ?    'slower' ?
>
> There are many items to consider in the optimization of filtering.
> -1- do you create the 300,000 lines yourself or inherit them
> -2- are the lines long strings or short (how many chartacters)
> -3- are the lines structured or more like descriptions or phrases
> -4- is the part to be filtered at the beginning or the end of each  
> line
> -5- there are numerous other considerations depending on the exact  
> task at hand
>
> Your request is far to vague to give definitive answers.
> As Mark Wieder said, please post some example lines of the data and  
> the code you are trying.  There are many innovative ways to use the  
> Rev chunking functions that make sure you get speed without  
> sacrificing accuracy ( false hits, false misses )
>
> Looking forward to more details.
> It is fun to consider the variations possibilities  :-)
>
> Jim Ault
> Las Vegas