60 hours divided by 60 is 2 minutes?
ambassador at fourthworld.com
Mon Oct 28 04:11:01 CST 2002
David Vaughan wrote:
>> I have a 5MB file with about 550000 lines that need to be processed by
>> script. A simple script that deletes a line if the previous line has
>> same contents. That takes more than 60 hours to complete. So I thought
>> divide the file into smaller files of about one 60th of the total
>> number of
>> lines. But instead of the expected hour of processing time, it took 2
>> minutes for each file to complete.
> I am a bit puzzled by your result in the first place. I generated
> 550000 lines with random data which had some chance of duplication in
> the next line. I then processed it to remove duplicates. The latter
> task took a whole four seconds. Not two minutes and not 60 hours; for
> the whole file, not for one sixtieth. Were you using "repeat for each"?
Also, when it comes to adding or deleting, working with arrays is much
faster than with large chunks. Remember that you can use arays and chunks
interhangeably with the split and combine commands.
Fourth World Media Corporation
Custom Software and Web Development for All Major Platforms
Developer of WebMerge 2.0: Publish any database on any site
Ambassador at FourthWorld.com http://www.FourthWorld.com
Tel: 323-225-3717 AIM: FourthWorldInc
More information about the use-livecode