Speeding up get URL

[Noel]

Yes, something like what you are describing could easily be confused 
with a DOS attack.

DOS attacks are done by flooding a server with requests for webpages 
to the point that the server crashes due to its inability to process 
all the requests.

Even if you are not considered a DOS attack, the company may not 
appreciate the bandwith that is being used for you to continually 
index their site and may deny your IP address access at some point.

  - Noel

At 10:38 AM 8/3/2008, you wrote:
>It's always one domain, the same domain, and I have no control over 
>the domain or its hosting company.  The domain itself probably has 
>millions of pages.  Anybody can sell products thru them, and they 
>make it very easy to do so.  So there are probably thousands (or 
>more) folks with massive quantities of products for sale.
>
>The worst thing will be if 40,000 products takes 4.5 hours (my last 
>two runs confirmed that), once it gets to hundreds of thousands it 
>could take *days*.  That would be bad.
>
>I'm having other options for partial runs, but there's always got to 
>be the ability for a full run.
>
>I'm not familiar with the ins and outs of a DOS attack, you mean 
>something like this could be confused for one?
>
>>Just a thought...  One factor might be if your list has the same domain
>>appearing as a contiguous block, the web server may be detecting that you
>>are not a human browsing and slow down the transfer rate.  One of my hosting
>>companies does this because they had bad experiences with denial of service
>>attacks.
>>
>>Hope this helps.
>>
>>Jim Ault
>>Las Vegas
>
>
>--
>   Dogs and bears, sports and cars, and patriots t-shirts
>   http://www.villagetshirts.com
>  WlND0WS and MAClNT0SH shareware games
>  http://www.gypsyware.com
>_______________________________________________
>use-revolution mailing list
>use-revolution at lists.runrev.com
>Please visit this url to subscribe, unsubscribe and manage your 
>subscription preferences:
>http://lists.runrev.com/mailman/listinfo/use-revolution