slow load URL

[Josh Mellicker]

This may not be the answer you expect, but using rsync to download  
might be your best option. It seems blazingly fast with tons of small  
files, partially because it automatically compresses before  
downloading and uncompresses afterwards.

Today, in a test, we downloaded a whole website with over 10,000  
files, 150 MB of data, in a couple minutes. (Actual data transferred  
was 57 MB)

It is a shell program, and I have not got it working with Rev yet, but  
one of the many shell gurus on this list may reveal the answer soon ;-)


On May 11, 2008, at 8:00 PM, Nicolas Cueto wrote:

> Hello,
>
> I have a stack that requires dowloading 40 to 200 image/sound files
> (about 40Kb each).  The download time is pretty long so I'm wondering
> if part of the problem is my script, which I've included below.
>
> Thank you.
>
> -- Nicolas Cueto
>
>
> on startDownloads
>  set the socketTimeoutInterval to 15000
>  put theUrl into tTheUrl
>  /*  Clear any cached urls.    */
>  repeat for each line thisURL in the cachedURLs
>     unload URL thisURL
>  end repeat
>  repeat for each line tImgOrSndID in tImgOrSndIDs
>      put tTheUrl & tImgOrSndID into tURLPath
>      load URL tURLPath
>      ShowInfo tDU
>      wait until "showInfo" is not among the items of the
> pendingMessages with messages
>  end repeat
> end startDownloads
>
>
> -- ShowInfo is a pending message which
> -- monitors download status every 10 milliseconds.
> -- By Eric Chatonet.
>
> on ShowInfo pUrl
>  local tStatus,tResult
>  -----
>  put URLStatus(pUrl) into tStatus
>  switch
>  case word 1 of tStatus = "error"
>    answer "Sorry: invalid URL or no connection."
>    put pUrl & cr after gFailedDownloads
>    break
>  case tStatus = empty
>    answer "Sorry: invalid URL or no connection."
>    put pUrl & cr after gFailedDownloads
>    break
>  case word 1 of tStatus = "timeout"
>    answer "Sorry: time out."
>    put pUrl & cr after gFailedDownloads
>    break
>  case item 1 of tStatus = "loading"
>      break
>  case word 1 of tStatus = "cached"
>    exit ShowInfo
>    break
>  end switch
>  send "ShowInfo" && pUrl to me in 10 milliseconds
> end ShowInfo
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your  
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution