background downloading alternatives (used to be: integrating rsync with Rev)

[Jim Ault]

You choice of text files for 'inter-standalone' communication is a good one.
Using sockets would only be worth while if you needed constant data passing
between the applications (standalones) and wanted to avoid hard drive
activity.  One way to optimize is to download to a second physical hard
drive while using the first hard drive for the text file writing and normal
computer operation.

Another reason to consider using sockets for your particular set of tasks is
to allow progress bars, error checking, or other updating tasks to be
reported to the main app and thus the user (eg stop downloading button in
the user app)  However, the helper app could just look for a 'command.txt'
file in a folder and, when present, read it, then delete it.

Sockets are fairly simple, especially when used on the same cpu.

In one very fast communication set of apps that I created, I programmed 3
apps to run on one computer and 2 apps to run on a second computer, all with
UDP sockets interconnecting them.  This allowed for use of multiple
processors (2 cpu's) and multi-threading via the operating system.  This was
used on a LAN (local area network) in conjunction with two computers in
separate offices.   This is a total of 4 cpu's, 8 apps, 3 networks.  The
system ran very smoothly for about 1.5 years until the project was
discontinued.

Hope this helps with the brain storming for optimal solutions.

Jim Ault
Las Vegas


On 5/30/08 11:06 AM, "Josh Mellicker" <josh at dvcreators.net> wrote:

> The whole reason for my rsync investigation is that I have a project
> that needs to download a lot of big files in the background, and allow
> the user to freely perform other activities during downloading. Using
> a libURL callback in the same engine has not worked for us, I believe
> it takes too much processing power from the engine and performance is
> bad to worse while downloading.
> 
> Our present course of action is to use one separate standalone just
> for downloading, and another standalone for the user to interact with.
> When files are needed, the main app creates a text file, then launches
> the downloader app. On openStack, the downloader.app goes to work.
> Once the list is done the downloader quits.
> 
> It is not ideal having to communicate with text files, but we have
> never tried socket communication, may tackle that later.
> 
> I have also looked into Bittorrent clients that can be operated with
> shell commands but this seems overcomplicated.
> 
> 
> On May 30, 2008, at 4:20 AM, Luis wrote:
> 
>> Hiya,
>> 
>> There is an easier install for rsync on Windows, called cwRsync,
>> from: www.itefix.no
>> 
>> Nexenta has a free GUI implementation of called rsyncshare: www.nexenta.com
>> 
>> There is another free to use cross platform option:
>> 
>> www.anyclient.com (Made by: www.jscape.com. They make the command
>> line 'FTCL' but it's a little steep at $299).
>> 
>> nnBackup is a Windows command line utility from: www.nncron.ru
>> 
>> WinMerge (www.winmerge.org) has a command line option.
>> 
>> In Windows XP (needs install from the XP Server Resource Kit, free
>> to download) and a default in Vista, you can use Robocopy, more info
>> here: www.ss64.com/nt/robocopy.html
>> I think it still doesn't do deltas like rsync, but it's been a while.
>> 
>> 
>> Cheers,
>> 
>> Luis.
>> 
>> 
>> 
>> On 30 May 2008, at 10:46, David Bovill wrote:
>> 
>>> Done a bit more research - rsync is available for Windows - but
>>> AFAIK it
>>> requires cygwin, whcih does mean that to distribute with Rev is not
>>> as
>>> simple as including a binary.
>>> 
>>> I've been looking at alternatives - the one I have used before on
>>> Linux and
>>> Windows, OSX with Rev is Unison - binaries are available. It's
>>> basically
>>> rsync, but lets you do it between two computers with an internet
>>> connection:
>>> 
>>>   - http://alan.petitepomme.net/projets/unison/index.html
>>> 
>>> I also found this programme which is Java - and can be run from
>>> commanline -
>>> so can be used by Rev. It looks like it does not require
>>> installation on the
>>> sserver and can do incremental backups to a NAS:
>>> 
>>>   - http://jfilesync.sourceforge.net/index.shtml
>>> 
>> 
>> _______________________________________________
>> use-revolution mailing list
>> use-revolution at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-revolution
> 
> _______________________________________________
> use-revolution mailing list
> use-revolution at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription
> preferences:
> http://lists.runrev.com/mailman/listinfo/use-revolution