revCopyFile chokes on 1,500 file loop

[Mark Schonewille]

Hi Sivakatirswami,

Revolution issues the revCopyFile command but doesn't wait for the  
result. You might try getting the result after executing the command,  
although I don't know whether this really helps. I suspect that the  
revCopyFile command uses AppleScript on Mac OS X. If an execution  
error occurs or if AppleScript simply doesn't respond because it is  
busy, possibly returning a time-out, Revolution doesn't notice this.

I haven't used this command for years now. I always read source files  
in binary format and write them to their destination: open files,  
read from source file, append to destination file, close files. If  
you do this in chunks (I mean data chunks, not batch chunks as you  
suggested), small enough to avoid blocking the interface and large  
enough to keep the speed high, you'll be surprised how fast  
Revolution can be. Do this in a repeat loop with messages or use the  
send in xx millisecs syntax. Works way better than revCopyFile.

Best regards,

Mark

--

Economy-x-Talk
Consultancy and Software Engineering
http://economy-x-talk.com
http://www.salery.biz

Download ErrorLib at http://economy-x-talk.com/developers.html and  
get full control of error handling in Revolution.



Op 4-aug-2006, om 10:57 heeft Sivakatirswami het volgende geschreven:

> platform-network context:
> Mac OSX client with  Rev app that is "talking to-about" files on an  
> OSX Server across the LAN.  All  path stems look like
>
> /Volumes/Server/WWW/*Sites/www.himalayanacademy.com/html/audio/"
>
> Goal: copy about 1,500 files from one set of directories to another  
> directory with a new name on the server, not on the client  
> machine,  which is simply initiating, controlling the process  
> remotely.
>
> I'm running this from a small rev card based database with about  
> 800 cards. Each card has the old path to the old file for 1 to 4  
> files and some fields from which to build a new file name. The  
> trimmed down script looks like this, where the Server  is mounted   
> on the desktop of the client machine via AFP.
>
> repeat with x = 1 to the number of  cards
>
> go to card x
>   # do stuff here to build a new file name and set up paths in  
> variables:
>
>    repeat for each line y in fld "soundFiles" # each record has a  
> fld w/list of  .wav, .ra, .mp3 files to step through
>       RevCopyFile (oldpath &"/"& oldFileName), (newpath & "/" & new  
> file name
>
> # an explicit instance if watching variable watcher  would  look  
> something like this:
> revCopyFile "/Volumes/Server/WWW/*Sites/www.himalayanacademy.com/ 
> html/audio/1998/soundclip.ra",\
> "/Volumes/Server/WWW/*Sites/www.himalayanacademy.com/html/audio/ 
> 1998/ad_1998-08-01_title.ra"
>
>   end repeat
> # next card....
>
> end repeat
>
> OK, simple enough and appears to work like a charm, and super fast  
> because no  files are being read and written by and from the client  
> machine,  but if we open the directory on the server to which files  
> are being copied to, we see whole sections of files are missing...  
> it's as if the script is  over-running the ability of the network 
> +server to  do the job.
>
> If I reduce the repeat look to something like
>
> repeat with x = 1 to 10
>
> go to card x
>   # do stuff here to build a new file name and set up paths in  
> variables:
>
>    repeat for each line y in fld "soundFiles" # each record has a  
> list of  .wav, .ra, .mp3 files I step through
>       RevCopyFile (oldpath &"/"& oldFileName), (newpath & "/" & new  
> file name
>   end repeat
>
> end repeat
>
> it works just fine, but if I just let it run from card 1 to 778,  
> the server can't handle it. Rev  rips  through the 778 cards about  
> as fast as it can update the video card to display the next card  
> and, in under 2 minutes it has thrown 1,500 requests at the server  
> to  copy and rename files ranging in size from 300K to 4 Megabytes.  
> Meanwhile we can hear the mac doing the successful copy beep sound  
> at a frantic pace, so much so that the  speaker can't keep up.
>
> I tried introducing a  wait" in the script,  but it  didn't help. I  
> assume from the docs that  Rev is just  doing a shell
>
> cp "old path-filename" "new path-filename"  in the background.  I'm  
> just guessing but I think we need to have some kind  of feedback in  
> the loop to make  sure the script is  not over running some limit  
> on the server which I don't know what  it  would be or how to get  
> that feedback data into the process. If you get the result on each  
> iteration, it is always empty, so that doesn't provide any progress  
> data. what we need is, algorithmically stated as if talking to the  
> server:
>
> copy this file  to that file
> Now  I give you some time
> You tell me if  you are ready?
> Can I issue another copy command? I'm  listening
> OK, yes I heard that, you are done, good
> copy next file to next file
>
>
> Any ideas? Of course I *can* do this  in chunks... like maybe set  
> the repeat for  x = 1 to 10 and then wait and do 11-20 etc... and  
> get the immediate job done, in probably less then 20 minutes, but  
> I'm interested in solving the problem because I could really use  
> this  functionality in other contexts for some massive global file  
> handling processes.
>
> TIA
>
> Sivakatirswami