Speeding up get URL

[Jim Sims]

On Aug 3, 2008, at 4:35 PM, Shari wrote:

> Goal:  Get a long list of website URLS, parse a bunch of data from  
> each page, if successful delete the URL from the list, if not put  
> the URL on a different list.  I've got it working but it's slow.  It  
> takes about an hour per 10,000 urls.  I sell tshirts.  Am using this  
> to create informational files for myself which will be frequently  
> updated.  I'll probably be running this a couple times a month and  
> expect my product line to just keep on growing.  I'm currently at  
> about 40,000 products but look forward to the day of hundreds of  
> thousands :-)  So speed is my need... (Yes, if you're interested my  
> store is in the signature, opened it last December :-)
>
> How do I speed this up?



I wonder if using "load" URL might be faster?


sims
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ClipaSearch Pro
http://www.einspine.com

Across Platforms - Code and Culture
http://www.ezpzapps.com/blog/