Web crawling/spidering with LiveCode?

[Keith Clarke]

Hi folks,
Having made some progress with screen-scraping specific web page content into LiveCode, I'm now wondering about how I might automate the process of finding the pages of interest on a website. 

The idea is to work through a documentation site (or wiki), returning structured tabular data containing key/value pairs, together with the the links to the relevant pages.

Has anyone experimented with creating website crawler/spidering procedures within LiveCode - or looked into accessing, parsing and processing site-map files?
Best,
Keith..