Recursive Directory Walker for really huge numbers of files

[Geoff Canyon]

At 7:41 PM -1000 5/8/02, Sivakatirswami wrote:
>    put whatFolder & "/" & x into line (the number of lines of gHierList)+1
>of gHierList

This is the line that is killing you. You're forcing Revolution to count the number of lines of the list each time you add a line (maybe twice, not sure).


This processes 10,000 files at about 400 files per second on my G3 400's boot drive:

global gHierList,gMainFolder,gBaseLevels
local tCount

on mouseUp
  put the ticks into tStart
  put "" into gHierList
  answer folder "Pick a folder you want to walk:"
  put it into gMainFolder
  if gMainFolder = "" then exit mouseUp
  set the itemDel to "/"
  put 0 into tCount
  put the number of items of gMainFolder into gBaseLevels
  directoryWalk gMainFolder
  put gHierList into field "result"
  put tCount && (tCount*60/(ticks()-tStart))
end mouseUp

on directoryWalk whatFolder
  --put whatFolder
  --exit to top
  set the itemDel to "/"
  set the directory to whatFolder
  put the files into temp
  add the number of lines of temp to tCount
  filter temp with "*.html"
  sort temp
  repeat for each line x in temp
    put whatFolder & "/" & x & cr after gHierList
  end repeat
  put the folders into tDirList
  sort tDirList
  delete line 1 of tDirList
  repeat for each line x in tDirList
    directoryWalk (whatFolder & "/" & x)
  end repeat
end directoryWalk
-- 

regards,

Geoff Canyon
gcanyon at inspiredlogic.com