process data in steps of 50 lines
userev at canelasoftware.com
Tue Jun 7 14:46:31 EDT 2011
On Jun 7, 2011, at 6:27 AM, Mark Schonewille wrote:
> Since the data is already in memory, there is no reason to process it in steps of 50. Also, using repear with x =... is very slow. Use repear for each with a counter instead:
I believe there is a major speed benefit to chunking the data into smaller groups. We optimized a data processing app with this technique and brought tasks from minutes of processing down to milliseconds.
The technique chunks the data into groups of 600 lines. Then you use a basic repeat. No with or for each. Then you read just the first line of the repeat. Then delete the first line of the dataset. You will see a major improvement in speed.
More information about the Use-livecode