XML & memory use/release

Jan Schenkel janschenkel at yahoo.com
Fri Jan 26 17:02:22 EST 2007


--- Martin Blackman <martinblackman at gmail.com> wrote:
> Dear listfolks,
> 
> I've noticed that creating an xml tree from a file
> using
> 'revcreatexmltree' can take up anything from 15 to
> 30 times the size
> of the file on disk in memory.  Does that sound like
> Rev is doing
> anything peculiar to those with more understanding
> of the underlying
> behaviour ?
> 
> I need to deal with some reasonably large files and
> so need to
> consider ways to avoid changing the user's computer
> into treacle. (
> I've been able to grind my 512MB laptop practically
> to standstill but
> my desktop seems to manage OK even when its physical
> memory available
> is very low)
> 
> [snip]
> 
> Any thoughts ?
> Many thanks
> Martin Blackman
> 

Hi Martin,

If you're handling huge XML files and don't actually
need to manipulate the data in the DOM tree (Document
Object Model), you could use the SAX approach (Simple
API for XML).
Rather than going through the entire file and building
up a huge tree structure in memory, SAX parsers send
messages as they traverse the XML text. Messages are
sent at the start and end of the parsing process, as
well as at the start abnd end of a tag, and the
element data.
So if you're just pulling information out of an XML
file, SAX parsers can save you a lot of memory.

In Revolution, you'll use a variation on the
'revCreateXMLTreeFromFile' function and trap the
'revXMLStartTree', 'revStartXMLNode',
'revStartXMLData', 'revEndXMLNode' and 'revXMLEndTree'
messages.
See the dictionary for more information on the
peculiarities of these messages. In a nutshell, when
you pass 'true' as the final parameter of the
'revCreateXMLTreeFromFile' function, the above
collection of messages will be sent to the current
card as it parses the XML structure.

I recently made a little stack that uses this approach
to 'pretty print' XML files that were produced by
another app and weren't meant for human consumption.
You can download this as a sample stack at the
following URL:
<http://www.quartam.com/tutorials/xmlprettyprint.rev>

For those who just want the script, here's what I put
in the card script:
##
on PrettyPrint pFilePath
  get
revCreateXMLTreeFromFile(pFilePath,false,false,true)
end PrettyPrint

on DisplayStatus pStatus
  put pStatus into field "fCurrentMessage"
end DisplayStatus

--

local sCurrentDepth

on revXMLStartTree
  DisplayStatus "revXMLStartTree"
  put "<?xml version='1.0'?>" into field
"fPrettyPrint"
  put 0 into sCurrentDepth
end revXMLStartTree

on revXMLEndTree
  DisplayStatus "revXMLEndTree"
  set the clipboardData["text"] to field
"fPrettyPrint"
  answer "Done..."
end revXMLEndTree

on revStartXMLNode pNodeName, pAttributes
  DisplayStatus "revStartXMLNode" && pNodeName
  add 1 to sCurrentDepth
  if char -1 of field "fPrettyPrint" is not return
  then put return after field "fPrettyPrint"
  repeat with i = 2 to sCurrentDepth
    put "  " after field "fPrettyPrint"
  end repeat
  put "<" & pNodeName after field "fPrettyPrint"
  repeat for each line tAttributeKVP in pAttributes
    put space & item 1 of tAttributeKVP & "=" after
field "fPrettyPrint"
    put quote & item 2 to -1 of tAttributeKVP & quote
after field "fPrettyPrint"
  end repeat
  put ">" & return after field "fPrettyPrint"
end revStartXMLNode

on revEndXMLNode pNodeName
  DisplayStatus "revEndXMLNode" && pNodeName
  
  then put return after field "fPrettyPrint"
  repeat with i = 2 to sCurrentDepth
    put "  " after field "fPrettyPrint"
  end repeat
  put "</" & pNodeName & ">" after field
"fPrettyPrint"
  subtract 1 from sCurrentDepth
end revEndXMLNode

on revStartXMLData pElementData
  DisplayStatus "revStartXMLData" && pElementData
  put word 1 to -1 of pElementData after field
"fPrettyPrint"
end revStartXMLData
##

Hope this helped,

Jan Schenkel.

Quartam Reports for Revolution
<http://www.quartam.com>

=====
"As we grow older, we grow both wiser and more foolish at the same time."  (La Rochefoucauld)


 
____________________________________________________________________________________
Do you Yahoo!?
Everyone is raving about the all-new Yahoo! Mail beta.
http://new.mail.yahoo.com



More information about the use-livecode mailing list