Uploading/Downloading stability

[Alex Tweedly]

Chipp Walters wrote:

> That's an interesting problem. I think I would approach it on two 
> fronts: the server side and the client side. I'll describe the 
> download method, you would reverse for upload.
>
> On the server side, I would break-up the file into smaller 
> chunks..perhaps use a Rev CGI to do it. Just read the whole binary 
> file into memory, binhex it, break it up into 30K chunks, and gzip 
> those to a folder with a unique naming sequence. Then create a 
> checkSum for each file and store it in a small text file.
>
> Then I'd just do 'get URL' from RR, download the text file, read the 
> checksums and filenames, and begin to download one after the other, 
> making sure the checksums match. Once they're all downloaded, then 
> combine them, and spit out the final 2Mb file.
>
> Not all that hard, just tedious. For upload, you may need to use 
> multipart encoding. Or, if you're posting to a Rev CGI, you could do 
> it with simple form data.
>
All good advice - as is th suggestion to use ftp and take advantage of 
its built-in resume capability - but I'm not sure it's enough.

> Richard said:
>
>>
>> My Rev program needs to upload and download a file of roughly 2 MB in 
>> size 3-8 times every 30 minutes via a wireless connection. This 
>> wireless connection is through the built-in airport card in an IBook. 
>> Sometimes this connection is received via a wired DSL or cable modem 
>> (and wireless router), but sometimes it feeds off a Powerbook 
>> connected to the net via a Verizon 3G PCMCIA card (using Apple's 
>> Internet Sharing capabilities).
>
I don't know if you're quite "on the edge" here, but you must be 
somewhere close.

wired DSL - let's guess you have 512k down and 128k up.
With 512k (bits) down (especially with a wifi hop in the end-to-end 
path), you will likely only get 30Kb(ytes) to 40Kb overall download 
speed (with http).

2Mb  @ 35Kb = approx 1 minute
8Mb @ 35Kb = approx 4 minutes

BUT - on upload, speed is (guessed) to be only 1/4 of that, so
2Mb - 8Mb = 4 to 16 minutes.

If you need to upload and download it - total of between 5 and 20 minutes.
And you need to do it 3-8 time = 15 minutes best case, to 160 minutes 
worst case, every 30 minutes.

I think you have a problem (even if there is no other use of that DSL 
line, and even if you don't lose a connection and need to repeat some 
fraction of a download).

Now it's true, you can do one upload and one download simultaneously, 
with (if you're lucky) relatively little impact on each other - but that 
may or may not fit in with the logic of the app. And you probably won't 
"be lucky" - you'll almost certainly see some interference between an 
upload and download at the same time.

So I think you should be looking at what can be done on each side 
(server and client) to reduce the data load
 - compression ?
 - recognize and trim out common data (i.e. does the entire 2Mb of data 
change each time ? can you do "delta" updates ?)
 - can you control the rate of download (i.e. is the "3-8" range in your 
control, or dictated by outside events) ?

-- 
Alex Tweedly       http://www.tweedly.net



-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.308 / Virus Database: 266.9.7 - Release Date: 12/04/2005