Has Anyone Got A Directory "Walker" Available

Alex Tweedly alex at tweedly.net
Thu May 3 20:00:08 EDT 2018


Thanks Ben - that's pretty similar to what I use. Avoiding recursion is 
(almost) always a good idea, if it can be done without jumping through 
hoops; for a file walker I timed it (a couple of years ago) at somewhere 
between 20 and 35% faster.

And it's so much easier to add a good progress indicator callback in the 
non-recursive method.

But I would really urge you to add the safety check that Richard 
recommended against protected-directories - otherwise you'll find 
yourself in an infinite loop.

-- Alex.

P.S. the other addition I have is a couple of parameters for 
"ignore-dot-folders and ignore-dot-files". I rarely want to see, for 
instance, .git folders, so in my library I default to ignoring 
directories beginning with a dot.


On 03/05/2018 20:48, Ben Rubinstein via use-livecode wrote:
> FWIW I don't use recursion for directory walking. My typical routine 
> (typing directly into email, may contain errors) is something along 
> the following lines:
>
>     -- assumes tRootFolder is the path to start reviewing
>     put return into tFoldersToDo
>     repeat until tFoldersToDo = empty
>
>         -- process another folder
>         put line 1 of tFoldersToDo into tSubFolder
>         delete line 1 of tFoldersToDo
>         set the defaultFolder to (tRootFolder & tSubFolder)
>
>         -- note sub-folders
>         repeat for each line f in the folders
>             if char 1 of f = "." then next repeat
>             put tSubFolder & "/" & f & return after tFoldersToDo
>         end repeat
>
>         -- process files
>         repeat for each line f in the files
>             if char 1 of f = "." then next repeat
>             -- do something with f
>             -- or store the path, i.e. tSubFolder & "/" & f
>         end repeat
>
>     end repeat
>
> HTH,
>
> Ben
>
> On 21/04/2018 09:15, Ali Lloyd via use-livecode wrote:
>> Ah, it will throw an error - so best to use try / catch around the 
>> function
>> calls.
>>
>> On Sat, Apr 21, 2018 at 9:13 AM Ali Lloyd <ali.lloyd at livecode.com> 
>> wrote:
>>
>>> Now that we have files(pFolder), folder(pFolder) (and files(pFolder,
>>> "detailed"), folders(pFolder, "detailed")), directory walking code 
>>> can be
>>> improved somewhat by not having to set the current folder.
>>>
>>> Actually, I haven't checked what happens when you get
>>> files(pRestrictedFolder)
>>>
>>>
>>> On Fri, Apr 20, 2018 at 7:49 PM Richard Gaskin via use-livecode <
>>> use-livecode at lists.runrev.com> wrote:
>>>
>>>> You may recall the dreaded recursion errors that have cropped up in
>>>> discussions of directory walkers.  They happen not because anyone has
>>>> directory trees > 40,000 folders deep, but because a permissions
>>>> restriction can prevent going into a subdirectory, causing the current
>>>> directory to be traversed again and again until the recursion error is
>>>> thrown.
>>>>
>>>> The way to avoid this is to add an error check when setting the
>>>> directory, e.g.:
>>>>
>>>>    set the directory to pFolder
>>>>    if the result is not empty then
>>>>      -- skip or report as needed
>>>>    end if
>>>>
>>>>
>>>> -- 
>>>>    Richard Gaskin
>>>>    Fourth World Systems
>>>>
>>>>
>>>> Sannyasin Brahmanathaswami wrote:
>>>>
>>>>   >  I found this in a toolbox. Sent by some ago, by a LiveCode deva
>>>>   >
>>>>   > For what it's worth :
>>>>   >
>>>>   > # Filters the strings "." and ".." from a list
>>>>   > function filterDots pList
>>>>   >    local tList
>>>>   >
>>>>   >    put pList into tList
>>>>   >    filter tList without "."
>>>>   >    filter tList without ".."
>>>>   >
>>>>   >    return tList
>>>>   > end filterDots
>>>>   > # Returns a filtered list of files in the current directory
>>>>   > function filteredFiles
>>>>   >    return filterDots(the files)
>>>>   > end filteredFiles
>>>>   >
>>>>   > # Returns a filtered list of folders in the current directory
>>>>   > function filteredFolders
>>>>   >    return filterDots(the folders)
>>>>   > end filteredFolders
>>>>   >
>>>>   > # Returns a list of files in the current directory including
>>>>   > # each file's full path.
>>>>   > function filteredFilesWithPaths
>>>>   >    local tFiles, tFilesWithPaths
>>>>   >
>>>>   >    put filteredFiles() into tFiles
>>>>   >    repeat for each line tFile in tFiles
>>>>   >       put the directory & slash & tFile & return after \
>>>>   >             tFilesWithPaths
>>>>   >    end repeat
>>>>   >    delete the last char of tFilesWithPaths
>>>>   >
>>>>   >    return tFilesWithPaths
>>>>   > end filteredFilesWithPaths
>>>>   >
>>>>   >
>>>>   > # Returns a list of files in a given folder, using recursion to
>>>>   > # include files in subfolders if desired.
>>>>   > function listFiles pFolder, pRecurse
>>>>   >    local tTotalFiles, tCurrentFiles, tFolders
>>>>   >
>>>>   > set the directory to pFolder
>>>>   >    put filteredFiles() into tCurrentFiles
>>>>   >    if not pRecurse then return tCurrentFiles
>>>>   >    if tCurrentFiles is not empty then
>>>>   >       put tCurrentFiles & return after tTotalFiles
>>>>   >    end if
>>>>   >    put filteredFolders() into tFolders
>>>>   >
>>>>   > repeat for each line tFolder in tFolders
>>>>   >       put listFiles((pFolder & slash & tFolder), pRecurse) into \
>>>>   >       tCurrentFiles
>>>>   >       if tCurrentFiles is not empty then
>>>>   >          put tCurrentFiles & return after tTotalFiles
>>>>   >       end if
>>>>   >    end repeat
>>>>   > delete the last char of tTotalFiles
>>>>   >
>>>>   >    return tTotalFiles
>>>>   > end listFiles
>>>>   >
>>>>   > # Returns a list of files with their containing folders, using
>>>>   > # recursion
>>>>   > # if desired to descend into sub folders.
>>>>   > function listFilesWithFolders pFolder, pRecurse
>>>>   >    local tTotalFiles, tCurrentFiles, tFolders
>>>>   >
>>>>   > set the directory to pFolder
>>>>   >    put filteredFiles() into tCurrentFiles
>>>>   >    if not pRecurse then return pFolder & return & "--" & return \
>>>>   >    & tCurrentFiles
>>>>   >    if tCurrentFiles is not empty then
>>>>   > put pFolder & return & "--" & return after tTotalFiles
>>>>   >       put tCurrentFiles & return & return after tTotalFiles
>>>>   >    end if
>>>>   > put filteredFolders() into tFolders
>>>>   >    repeat for each line tFolder in tFolders
>>>>   >       put listFilesWithFolders((pFolder & slash & tFolder), \
>>>>   >       pRecurse) into tCurrentFiles
>>>>   >       if tCurrentFiles is not empty then put tCurrentFiles & \
>>>>   >       return after tTotalFiles
>>>>   > end repeat
>>>>   >    delete the last char of tTotalFiles
>>>>   >
>>>>   >    return tTotalFiles
>>>>   > end listFilesWithFolders
>>>>   >
>>>>   > # Returns a list of files with the full paths
>>>>   > function listFilesWithPaths pFolder, pRecurse
>>>>   >    local tTotalFiles, tCurrentFiles, tFolders
>>>>   >
>>>>   >    set the directory to pFolder
>>>>   >    put filteredFilesWithPaths() into tCurrentFiles
>>>>   >    if not pRecurse then return tCurrentFiles
>>>>   > if tCurrentFiles is not empty then put tCurrentFiles & \
>>>>   >    return after tTotalFiles
>>>>   >    put filteredFolders() into tFolders
>>>>   >    repeat for each line tFolder in tFolders
>>>>   >       put listFilesWithPaths((pFolder & slash & tFolder), \
>>>>   >  pRecurse) into tCurrentFiles
>>>>   >       if tCurrentFiles is not empty then put tCurrentFiles & \
>>>>   >       return after tTotalFiles
>>>>   > end repeat
>>>>   >    delete the last char of tTotalFiles
>>>>   >
>>>>   >    return tTotalFiles
>>>>   > end listFilesWithPaths
>>>>   >
>>>>   > # Returns a sorted list of the files in pFolder. Again using
>>>>   > # recursion if
>>>>   > # required.
>>>>   > function listSortedFiles pFolder, pRecurse
>>>>   >    local tFiles
>>>>   >
>>>>   >    put listFiles(pFolder, pRecurse) into tFiles
>>>>   >    sort lines of tFiles by listSortedFilesSortKey(each)
>>>>   >
>>>>   >    return tFiles
>>>>   > end listSortedFiles
>>>>   >
>>>>   > # Used by the listSortedFiles() function. Returns a sort key
>>>>   > # from each file's name to
>>>>   > # allow the sorting to be fully customized.
>>>>   > function listSortedFilesSortKey pFile
>>>>   >    # Use this value for a normal text sort
>>>>   >    return pFile
>>>>   > end listSortedFilesSortKey
>>>>   >
>>>>   >
>>>>   > on doSomething pFile
>>>>   >    put pFile & return after msg
>>>>   > end doSomething
>>>>   >
>>>>   > --This will achieve similar results to listing the files except 
>>>> that
>>>>   > it will be slower because text has to be drawn to the screen at 
>>>> each
>>>>   > iteration.
>>>>   >
>>>>   > # Use this template function to perform an action on each file
>>>>   > # in a folder
>>>>   > on doForEachFile pFolder, pRecurse
>>>>   >    set the directory to pFolder
>>>>   >    repeat for each line tFile in filteredFiles()
>>>>   >       doSomething (pFolder & slash & tFile)
>>>>   >    end repeat
>>>>   >    if pRecurse then
>>>>   >       repeat for each line tFolder in filteredFolders()
>>>>   >          doForEachFile (pFolder & slash & tFolder), pRecurse
>>>>   >       end repeat
>>>>   >    end if
>>>>   > end doForEachFile
>>>>   >
>>>>   >
>>>>   >
>>>>   >     I have directory available this days of Ken Ray, in the on 
>>>> system
>>>>   > of set the default folder and get the the files, registered 
>>>> that path.
>>>>   > descend recursively to the level… etc.
>>>>   >
>>>>   >     Do anyone has a new one handy they could share using the 
>>>> folder
>>>>   > and file function?
>>>>   >
>>>>   >     BR
>>>>   >
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> use-livecode mailing list
>>>> use-livecode at lists.runrev.com
>>>> Please visit this url to subscribe, unsubscribe and manage your
>>>> subscription preferences:
>>>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>>
>>>
>> _______________________________________________
>> use-livecode mailing list
>> use-livecode at lists.runrev.com
>> Please visit this url to subscribe, unsubscribe and manage your 
>> subscription preferences:
>> http://lists.runrev.com/mailman/listinfo/use-livecode
>>
>
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your 
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode





More information about the use-livecode mailing list