LibURL Error Previous request not completed

[Alex Tweedly]

Dave Cragg wrote:

>
> On 2 Aug 2006, at 16:50, Mark Schonewille wrote:
>
>>
>> FTP servers don't care much about the number of connections. Good  
>> FTP clients upload and download files at the same time, using  
>> different connections.
>
>
> I have to question whether there are any benefits in doing that.  
> Whether you download files sequentially or out of sync, the server is  
> trying to do essentially the same thing -- push the same data down  
> the same physical pipe. Does it matter the order in which it does it?
>
Yes, it does matter. There are a number of cases where multiple 
connections will win out :

1. avoiding start/finish round-trip delays for each transfer.
      You don't of course avoid them - but you can interleave them with 
the parallel transfers, and hence keep the pipe closer to full.
2. transient congestion (and packet-drops) have less effect on multiple 
connections
      Esp. if the congested router is using RED
3. rate-limiting per connection from the server 
      can be imposed lower than your connection bandwidth
4. bandwidth allocation on congested links
      Esp if the upstream router is using WFQ or DRR queuing, and high 
capacity connections are severely limited
5. .... many more .....

> But when establishing new ftp connections, there is an overhead for  
> making the socket connection and negotiating the login. If you are  
> uploading/downloading a lot of small files, that overhead may become  
> proportionally high and actually slow things down.

Indeed - you'd want to establish the ftp connection(s) once, and 
allocate individual transfers between them, not start a new connection 
per file.

>
> I don't have any data to show which approach might be faster. Do you  
> have any pointers?
>
Nothing interesting I can share, sorry.
There are a few public ones on GridFTP, such
www.slac.stanford.edu/econf/C0303241/proc/papers/TUCP008.PDF

I also found
http://www.applelinks.com/index.php/more/charles_moore_reviews_the_captain_ftp_445_ftp_client/
though I haven't read it thoroughly yet.

> For what it's worth, the client I usually use (Fetch) uploads/ 
> downloads multiple files sequentially over a single connection.
>


-- 
Alex Tweedly       http://www.tweedly.net

-------------- next part --------------
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.394 / Virus Database: 268.10.5/406 - Release Date: 02/08/2006