recursion limits

[Alex Tweedly]

Chris Sheffield wrote:

> I know recursion has been discussed in the past, and I'm wondering if  
> anyone has ever run into any limits (i.e. memory problems) with  
> recursion in Rev.  I am working on a little backup utility for my own  
> use, and I'm wondering what would happen if I decided to back up my  
> entire hard drive?  Would Rev choke on that?  I realize it could  
> potentially take hours.  Would I start getting out of memory errors?   
> The utility uses a directory walking function to create a list of all  
> sub folders and files to be backed up.
>
> Anyone have some detailed results with this type of thing?

I doubt that you'd run into recursion limits walking a disk (unless you 
have exceptionally large disks, or deeply nested directories) - but it's 
better to to avoid the possibility than to get a nasty surprise 3 hours 
into a backup ....

Two suggestions.

1. Two phases. Generate a complete list of files *first*, then operate 
on them. This has two advantages; the resources for recursion aren't 
needed at the same time as the resources for the real work, and more 
importantly - if there are problems with limits, you find out in the 
first few minutes, before you have wasted much time.

A directory walk simply storing file names runs at about 2000 files per 
second (on my slow laptop) up to about 7000 per second on a moderate 
desktop.  (Obviously very dependent on the directory structure).

2. Iterate instead of recurse. Change your recursive call to a "send ... 
to me in 0 secs", and check the pendingmessages to tell when to exit (by 
"send .... to another"). Watch out for the need to quote directory names 
(I have directory names containing "comma" that confused my first 
attempt to do this).

This has no worries about limits on recursion. (Are there limits on the 
number of pending messages ? Or performance issues with (tens of) 
thousands of pending messages ? Hmmmmm).

This ran slower half the speed of the simple recursive method for me. 
Could probably be optimized if needed.

Personally I'd just do the first of these - but the second one was an 
interesting exercise .....


-- 
Alex Tweedly       http://www.tweedly.net



-- 
No virus found in this outgoing message.
Checked by AVG Anti-Virus.
Version: 7.0.338 / Virus Database: 267.10.0/63 - Release Date: 03/08/2005