Divide Large Data Blob?

[Richard Gaskin]

Rick Harrison wrote:

> I have a large chunk of data that I want to
> search as quickly as possible.  
> 
> Unfortunately the part I want to search is the 
> middle third of the data.  The other thirds at 
> the beginning and at the end are just junk and 
> slow down my search so I want to get rid of them.
> 
> I don’t want to search line by line as that
> takes way too long.
> 
> There’s no unique character dividing any
> of these data regions.
> 
> What’s the best way to do this?

The offset function has an optional third param for starting point, so 
setting that value to the number of bytes you want to skip would take 
you right where you want to be. Offset can then be used in a loop, 
updating the starting value as you go. Exiting the loop once your into 
the tail you don't want to index can be a simple if to escape the loop.

There may also be options for using delimiters. What does the source 
data look like, and what should the resulting index look like?

And the most central question with things like this: how large is "large"?

--
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  Ambassador at FourthWorld.com                http://www.FourthWorld.com