60 hours divided by 60 is 2 minutes?
dvk at dvkconsult.com.au
Mon Oct 28 03:44:01 EST 2002
On Friday, Oct 25, 2002, at 21:35 Australia/Sydney, MultiCopy
> Hi all,
> I have a 5MB file with about 550000 lines that need to be processed by
> script. A simple script that deletes a line if the previous line has
> same contents. That takes more than 60 hours to complete. So I thought
> divide the file into smaller files of about one 60th of the total
> number of
> lines. But instead of the expected hour of processing time, it took 2
> minutes for each file to complete.
I am a bit puzzled by your result in the first place. I generated
550000 lines with random data which had some chance of duplication in
the next line. I then processed it to remove duplicates. The latter
task took a whole four seconds. Not two minutes and not 60 hours; for
the whole file, not for one sixtieth. Were you using "repeat for each"?
> I understand processes are faster with less data in memory, but I never
> would have thought the difference would be this big.
> Any thoughts on how this is possible and what we can learn from it when
> making programs?
> use-revolution mailing list
> use-revolution at lists.runrev.com
More information about the Use-livecode