60 hours divided by 60 is 2 minutes?

[David Vaughan]

On Friday, Oct 25, 2002, at 21:35 Australia/Sydney, MultiCopy 
Rotterdam-Zuid wrote:

> Hi all,
>
> I have a 5MB file with about 550000 lines that need to be processed by 
> a
> script. A simple script that deletes a line if the previous line has 
> the
> same contents. That takes more than 60 hours to complete. So I thought 
> I
> divide the file into smaller files of about one 60th of the total 
> number of
> lines. But instead of the expected hour of processing time, it took 2
> minutes for each file to complete.

Terry

I am a bit puzzled by your result in the first place. I generated 
550000 lines with random data which had some chance of duplication in 
the next line. I then processed it to remove duplicates. The latter 
task took a whole four seconds. Not two minutes and not 60 hours; for 
the whole file, not for one sixtieth. Were you using "repeat for each"?

regards
David
>
> I understand processes are faster with less data in memory, but I never
> would have thought the difference would be this big.
>
> Any thoughts on how this is possible and what we can learn from it when
> making programs?
>
> Terry
>
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> http://lists.runrev.com/mailman/listinfo/use-revolution
>