How to filter a big list

[Jim Ault]

On Oct 19, 2009, at 1:49 PM, Jérôme Rosat wrote:

> I wish to filter a list which contains approximately 300'000 lines.  
> I try the "filter ... with" command. It's slow.
>
> I try the "Repeat for each" loop but it's slower.
>
> Is it possible to use the "filter ... with" command and to "force"  
> RunRev to check only one "item" of the line and not the whole line  
> with the "filterPattern" ?
>
>


First, what do you mean by 'slow' ?    'slower' ?

There are many items to consider in the optimization of filtering.
-1- do you create the 300,000 lines yourself or inherit them
-2- are the lines long strings or short (how many chartacters)
-3- are the lines structured or more like descriptions or phrases
-4- is the part to be filtered at the beginning or the end of each line
-5- there are numerous other considerations depending on the exact  
task at hand

Your request is far to vague to give definitive answers.
As Mark Wieder said, please post some example lines of the data and  
the code you are trying.  There are many innovative ways to use the  
Rev chunking functions that make sure you get speed without  
sacrificing accuracy ( false hits, false misses )

Looking forward to more details.
It is fun to consider the variations possibilities  :-)

Jim Ault
Las Vegas