process data in steps of 50 lines

Mark Talluto userev at
Tue Jun 7 14:46:31 EDT 2011

On Jun 7, 2011, at 6:27 AM, Mark Schonewille wrote:

> Since the data is already in memory, there is no reason to process it in steps of 50. Also, using repear with x =... is very slow. Use repear for each with a counter instead:

I believe there is a major speed benefit to chunking the data into smaller groups.  We optimized a data processing app with this technique and brought tasks from minutes of processing down to milliseconds.

The technique chunks the data into groups of 600 lines.  Then you use a basic repeat.  No with or for each.  Then you read just the first line of the repeat.  Then delete the first line of the dataset.  You will see a major improvement in speed.

Best regards,

Mark Talluto

More information about the Use-livecode mailing list