Speeding up get URL

Jim Sims sims at ezpzapps.com
Sun Aug 3 11:27:10 EDT 2008

On Aug 3, 2008, at 4:35 PM, Shari wrote:

> Goal:  Get a long list of website URLS, parse a bunch of data from  
> each page, if successful delete the URL from the list, if not put  
> the URL on a different list.  I've got it working but it's slow.  It  
> takes about an hour per 10,000 urls.  I sell tshirts.  Am using this  
> to create informational files for myself which will be frequently  
> updated.  I'll probably be running this a couple times a month and  
> expect my product line to just keep on growing.  I'm currently at  
> about 40,000 products but look forward to the day of hundreds of  
> thousands :-)  So speed is my need... (Yes, if you're interested my  
> store is in the signature, opened it last December :-)
> How do I speed this up?

I wonder if using "load" URL might be faster?

ClipaSearch Pro

Across Platforms - Code and Culture

More information about the use-livecode mailing list