Speeding up get URL

[Shari]

It's always one domain, the same domain, and I have no control over 
the domain or its hosting company.  The domain itself probably has 
millions of pages.  Anybody can sell products thru them, and they 
make it very easy to do so.  So there are probably thousands (or 
more) folks with massive quantities of products for sale.

The worst thing will be if 40,000 products takes 4.5 hours (my last 
two runs confirmed that), once it gets to hundreds of thousands it 
could take *days*.  That would be bad.

I'm having other options for partial runs, but there's always got to 
be the ability for a full run.

I'm not familiar with the ins and outs of a DOS attack, you mean 
something like this could be confused for one?

>Just a thought...  One factor might be if your list has the same domain
>appearing as a contiguous block, the web server may be detecting that you
>are not a human browsing and slow down the transfer rate.  One of my hosting
>companies does this because they had bad experiences with denial of service
>attacks.
>
>Hope this helps.
>
>Jim Ault
>Las Vegas


-- 
   Dogs and bears, sports and cars, and patriots t-shirts
   http://www.villagetshirts.com
  WlND0WS and MAClNT0SH shareware games
  http://www.gypsyware.com