Arrays: new and old keys, i

Trevor DeVore lists at mangomultimedia.com
Thu Sep 11 15:57:35 EDT 2008


On Sep 11, 2008, at 3:10 PM, David Bovill wrote:

>> 1) Ability to reference an multi-dimensional keys dynamically.  
>> Right now we
>> have to build the array key reference and then use 'do'.
>
> Yes - that was the first problem I came across. I wanted to write a
> recursive function, and since you don't know how deep you will need  
> to go
> you have to use "do" which destroys the reason to use arrays 9ie  
> speed) in
> the first place. This makes it useless to use as a native data  
> structure for
> things like arrays - or at least impossible to create generic  
> libraries for
> tree / xml data structures.

Generic conversion of XML to Array and Array to XML is possible  
without dynamic keys once you have decided on a way to represent the  
XML as an array. The XML conversion routines in the stack I provide  
with the article can convert between XML and arrays and don't need  
dynamically generated keys. I'm using the XML to Array conversion  
routines extensively right now with web services and have been quite  
happy with them.

I've attached latest version of these handlers at the bottom of this  
email for reference. The entry points are ConvertArrayToXML() and  
ConvertXMLToArray(). Look at ConvertXMLNodeToArray() for an example of  
the recursive call.

I have another scenario where I had to resort to 'do' though. I'm  
converting SQL queries to a hierarchal array but unlike XML SQL  
results have no sense of hierarchy. So I have a couple of 'do'  
statements in the code which I will promptly replace when/if the  
engine is updated to support dynamic keys.

>> 2) Ability to pass a key of an array by reference. Currently you  
>> can't do
>> this:
>
> OK. But "theArray[1]" is just another array - so surely you can do  
> this:
>
> on mouseUp
>   put "Hi There" into theArray[1]["name"]
>   put theArray[1] into firstArray
>   DoSomething firstArray
> end mouseUp
>
> command DoSomething @pArrayA
>   ....
> end DoSomething
>
>  Which I almost prefer for legibility - so that's no biggy for me?

Right, not a huge deal. But you hate to have to make an extra copy  
when dealing with large data sets and it is extra lines of code. You  
left off one line of the code though. You need to put firstArray back  
into theArray[1] after calling DoSomething in order to get a pass by  
ref equivalent:

put theArray[1] into firstArray
DoSomething firstArray
put firstArray into theArray[1]

>> 3) Ability to reference elements of an array in the order they were  
>> added to the array:
>
> Totally. I posted about this earlier. I have to do a lot of  
> scripting to get
> around this


Yes, it is unfortunate that we have to resort to custom sorts (see  
SortArrayKeysWithXMLOrdering in XML code) or numeric sorts in order to  
iterate through keys sequentially. Hopefully that will be addressed in  
the near future.

Example of iterating sequentially through a numerically keyed array:

put the keys of theArrayA into theKeys
sort lines of theKeys numeric
repeat for each line theIndex in theKeys
    ....
end repeat

All that being said the new arrays have been a huge productivity boost  
on my end. Code is running faster, is easier to read and easier to  
write.


Regards,

-- 
Trevor DeVore
Blue Mango Learning Systems
ScreenSteps: http://www.screensteps.com
Developer Resources: http://revolution.bluemangolearning.com



--
-- Converts an XML tree into a Revolution multi-dimensional array.
-- A nodes attributes will be stored as an array of it's "@attributes"  
key.
-- Node names will retain the sequence information (i.e. node[1],  
node[2], etc.).
-- This information is necessary to determine order that keys should  
be processed in. Example:
-- set the itemDelimiter to "["
-- put the keys of theArray into theKeys
-- sort theKeys numeric by the last item of each
--
-- pUseValueKey: The default value is false. In this case you get an  
array that has an @attributes
-- key for nodes that have attributes and either a) no value or b)  
only child nodes. Otherwise it contains the node contents.
-- Set to true if you want to store a nodes value in the '@value' key.  
This will allow a key to have
-- both attributes (in @attributes key) and a value (in @value key).
--
function ConvertXMLToArray pXML, pStoreEncodedAs, pUseValueKey
     local theArray,theResult,theRootNode,theTreeID
     local theXMLEncoding

     ## Create an XML tree from XML text
     put revCreateXMLTree(pXML, true, true, false) into theTreeID

     if theTreeID is an integer then
         ## Determine the encoding of the XML, default to UTF-8
         put matchtext(pXML, "<\?xml (.*)encoding=" & quote & "(.*)" &  
quote & "\?>", versionMatch, theXMLEncoding) into theResult
         if theXMLEncoding is empty then put "utf-8" into theXMLEncoding

         ## Now convert to array.
         ## The 1st dimension has one key which is the name of the  
root node.
         put revXMLRootNode(theTreeID) into theRootNode
         if theRootNode is not empty and not(theRootNode begins with  
"xmlerr,") then
             put ConvertXMLNodeToArray(theTreeID, theRootNode,  
theXMLEncoding, pStoreEncodedAs, pUseValueKey) into  
theArray[theRootNode]
         end if
     end if

     return theArray
end ConvertXMLToArray


function ConvertXMLTreeToArray pXMLTree, pStoreEncodedAs, pUseValueKey
     return ConvertXMLToArray(revXMLText(pXMLTree), pStoreEncodedAs,  
pUseValueKey)
end ConvertXMLTreeToArray


--
-- Converts a multi-dimensional array to an XML tree.
-- The array should contain one key in the 1st dimension which
-- will become the root node. Attributes of a node should be stored
-- as an array in the @attributes key. Sequence information for multiple
-- nodes with the same name should be included in the node name using
-- brackets (i.e. node[1], node[2], node[3]).
-- Returns an xml tree id (integer) or an error message.
--
function ConvertArrayToXML pArray, pArrayEncoding, pStoreEncodedAs
     local theError,theRootNode,theXML,theXMLTree

     ## if pArrayEncoding is empty then current platform encoding is  
assumed
     if pStoreEncodedAs is empty then put "UTF-8" into pStoreEncodedAs

     ## Create XML for root node. Note that we take extra steps in  
order to support
     ## converting an array that only represents part of a tree rather  
than the entire tree.
     ## In this case there may be multiple nodes at the root level.
     put line 1 of the keys of pArray into theRootNode
     set the itemdelimiter to "["
     put "<" & item 1 of theRootNode & "/>" into theXML

     ## Create XML needed to create tree
     put format("<?xml version=\"1.0\" encoding=\"%s\"?>%s", \
             pStoreEncodedAs, theXML) into theXML
     put revCreateXMLTree(theXML, true, true, false) into theXMLTree

     if theXMLTree is an integer then
         ## Loop over all nodes at root level
         put SortArrayKeysWithXMLOrdering(pArray) into theNodes

         ## Create tree using helper function
         repeat for each line theNode in theNodes
             ConvertArrayDimensionToXML pArray[theNode], theXMLTree,  
slash & theNode, \
                     pArrayEncoding, pStoreEncodedAs
             put the result into theError

             if theError is not empty then exit repeat
         end repeat

         if theError is not empty then
             ## something went wrong, clean bad tree
             revDeleteXMLTree theXMLTree
         end if
     else
         put theXMLTree into theError
     end if

     if theError is not empty then
         return theError
     else
         return theXMLTree
     end if
end ConvertArrayToXML

--
-- Helper function for ConvertArrayToXML
-- Converts the multi-dimensional array pArray to nodes in pTreeID. 
-- Calls itself recursively.
-- Returns error message.
--
private command ConvertArrayDimensionToXML pArray, pTreeID, pNode,  
pArrayEncoding, pStoreEncodedAs
     local theError,theKey,theKeys,theNode

     ## A workaround for fact that Revolution does not return
     ## keys in the order we created them
     put SortArrayKeysWithXMLOrdering(pArray) into theNodes

     ## Arrays might have sequencing info in name
     ## (i.e. step[1], step[2], ... )
     set the itemdelimiter to "["

     repeat for each line theFullNode in theNodes
         put item 1 of theFullNode into theNode

         ## Look for attributes. These will be added as attributes to  
pNode.
         if theNode is "@attributes" then
             repeat for each line theKey in the keys of  
pArray[theFullNode]
                 revSetXMLAttribute pTreeID, pNode, theKey, \
                         EncodeString(pArray[theFullNode][theKey], \
                         pArrayEncoding, pStoreEncodedAs)
                 if the result begins with "xmlerr," then
                     put the result && "(setting attribute" && theKey  
&& "for node" && pNode & ")" into theError
                 end if

                 if theError is not empty then exit repeat
             end repeat

         else if theNode is "@value" then
             ## This XML tree is using complex structure. Node is the  
value of the parent node
             revPutIntoXMLNode pTreeID, pNode,  
EncodeString(pArray[theFullNode], pArrayEncoding, pStoreEncodedAs)
             if the result begins with "xmlerr," then
                 put the result && "(adding child node" && theNode &&  
"to node" && pNode & ")" into theError
             end if

         else
             if the keys of pArray[theFullNode] is not empty then
                 ## Node has children. Add node to XML tree then call  
self recursivly to create children nodes.
                 revAddXMLNode pTreeID, pNode, theNode, empty
                 if the result begins with "xmlerr," then
                     put the result && "(adding node" && theNode & ")"  
into theError
                 end if

                 if theError is empty then
                     ConvertArrayDimensionToXML pArray[theFullNode],  
pTreeID, pNode & slash & theFullNode, \
                             pArrayEncoding, pStoreEncodedAs
                     put the result into theError
                 end if
             else
                 ## Node has no children but possibly a value. Create  
node and add value (which may be empty).
                 revAddXMLNode pTreeID, pNode, theNode, \
                         EncodeString(pArray[theFullNode],  
pArrayEncoding, pStoreEncodedAs)
                 if the result begins with "xmlerr," then
                     put the result && "(adding child node" && theNode  
&& "to node" && pNode & ")" into theError
                 end if
             end if
         end if

         if theError is not empty then exit repeat
     end repeat

     return theError
end ConvertArrayDimensionToXML


--
-- Revolution array keys are never guaranteed to be in order you  
created 
-- them in so we must come up with some other way of maintaining 
-- proper sequence. For arrays representing XML, the XML syntax is 
-- used (i.e. node[1], node[2], etc.). This handler will sort keys  
that use 
-- this syntax for representing sequence.
--
function SortArrayKeysWithXMLOrdering pArray
     put the keys of pArray into theKeys
     set the itemdelimiter to "["
     sort theKeys numeric by the last item of each -- 1], 2], 3], etc.

     set the wholematches to true
     put lineoffset("@attributes", theKeys) into theLineNo
     if theLineNo > 0 then
         delete line theLineNo of theKeys
     end if

     return theKeys
end SortArrayKeysWithXMLOrdering


-- 
-- Helper function for ConvertXMLToArray.
-- Converts an XML node to a multi-dimensional array. 
-- Calls itself recursively.
--
private function ConvertXMLNodeToArray pTreeID, pNode,  
pXMLTreeEncoding, pStoreEncodedAs, pUseValueKey
     local theArrayA,theAttributes,theChildNode,theKey

     ## Look for attributes of the node. Store as array in  
"@attributes" key
     put revXMLAttributes(pTreeID, pNode, tab, cr) into theAttributes
     if theAttributes is not empty then
         put EncodeString(theAttributes, pXMLTreeEncoding,  
pStoreEncodedAs) into theAttributes
         split theAttributes by cr and tab -- create array
         put theAttributes into theArrayA["@attributes"]
     end if

     ## Look for children nodes.
     set the itemdelimiter to slash
     put revXMLFirstChild(pTreeID, pNode) into theChildNode
     if theChildNode is empty or theChildNode begins with "xmlerr," then
         put EncodeString(revXMLNodeContents(pTreeID, pNode),  
pXMLTreeEncoding, pStoreEncodedAs) into theValue
         if word 1 to -1 of theValue is empty and the keys of  
theArrayA is not empty then
             ## Empty node that has attributes
             return theArrayA
         else if pUseValueKey then
             ## Force value into @value
             put theValue into theArrayA["@value"]
             return theArrayA
         else
             ## Single Node with value: Return value. Attributes are  
ignored.
             return theValue
         end if
     else
         ## Child nodes were found. Recursively call self and store  
result in array.
         repeat while theChildNode is not empty and not (theChildNode  
begins with "xmlerr,")
             put the last item of theChildNode into theKey
             put ConvertXMLNodeToArray(pTreeID, theChildNode,  
pXMLTreeEncoding, pStoreEncodedAs, pUseValueKey) into theArrayA[theKey]
             put revXMLNextSibling(pTreeID, theChildNode) into  
theChildNode
         end repeat

         return theArrayA
     end if
end ConvertXMLNodeToArray

--
-- Helper function for converting the encoding of strings when  
converting to and from XML.
--
private function EncodeString pString, pInEncoding, pOutEncoding
     ## convert utf-8 to utf8 for uniencode/decode
     replace "-" with empty in pInEncoding
     replace "-" with empty in pOutEncoding

     if pInEncoding is not empty then
         -- if pOutEncoding is empty then pString will be converted to  
the current platform encoding
         return unidecode(uniencode(pString, pInEncoding), pOutEncoding)
     else
         if pOutEncoding is not empty then
             -- if pInEncoding is empty then pString is assumed to be  
in the current platform encoding
             return unidecode(uniencode(pString, pInEncoding),  
pOutEncoding)
         else
             return pString
         end if
     end if
end EncodeString



More information about the use-livecode mailing list