Divide Large Data Blob?

[Bob Sneidar]

Do you know exactly which lines you need to toss, or do you need to searc the data to find out where the beginning and end of the useful data is? 
If the former, then just put line x to y of your data into a new variable. If the latter, then a divide and conquer approach might be the answer. Get the line 30% in, test for valid, get the line 40% in, test, then 35% then 32.5% or 37.5% depending on your test. 

You may only have to do this a dozen or so times to find the exact line where your valid data begins. 

The other way of course is to get it all into a SQL database (how did you all know I was going to say that??) The downside is that you have to iterate through all your data once. The upside is a good one liner query statement may be all you need to process your data. And if you need to make multiple passes at your data, all the better. 

Bob S

> On May 16, 2022, at 10:46 , Rick Harrison via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> I have a large chunk of data that I want to
> search as quickly as possible.  
> 
> Unfortunately the part I want to search is the 
> middle third of the data.  The other thirds at 
> the beginning and at the end are just junk and 
> slow down my search so I want to get rid of them.
> 
> I don’t want to search line by line as that
> takes way too long.
> 
> There’s no unique character dividing any
> of these data regions.
> 
> What’s the best way to do this?
> 
> Thanks in advance!
> 
> Rick
> 
> 
> 
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode