Maintain Order in JSON Import

[Richard Gaskin]

Sannyasin Brahmanathaswami wrote:

 > On 9/14/16, 11:28 AM, Richard Gaskin wrote:
 >
 >     Do you need to round-trip?  Where does the data come from, and
 >     where is it going?
 >
 >     If any part of that is LC on both ends you can save _much_ time
 >     for that using an encoded array.

 > The use case is cross session data on mobile where we would like to
 > be able to examine/change the "array" in a text editor.

Have you considered YAML?

JSON was designed as an optimal solution for serialization in 
environments using the JavaScript interpreter.  While it is a plain text 
format human readability/writability is a by-product, with the primary 
goal being exchange of data with REST services that can be de-serialized 
in a call to JS' eval function.

YAML was designed to provide similar serialization to/from plain text, 
but optimized for human readability/writability.  Mark Wieder has a nice 
pair of functions for using it in LC, converting to and from arrays.


 > The only way to save an encoded array would be to insert is a a
 > custom property in an LC stack and save to the writeable folder
 > on mobile.

You can indeed store any data, text or binary in custom properties.  But 
LiveCode also has built-in options for reading/writing both text and 
binary data in files.

The arrayEncode function converts the collection of pointers and memory 
buckets associated with an array into one reasonably compact binary 
form, as easily handled with any file or network I/O routines as any 
other binary data, like images.

You can store LSON in a custom property, but you can also store it in a 
file by itself:

   put arrayEncode(tArray) into url "binfile:/path/to/file.lson"

To read it back into an array:

   put arrayDecode(url "binfile:/path/to/file.lson") into tArray


Just as JSON is designed hand-in-glove to be optimized for the 
JavaScript interpreter, LSON is the with-the-grain serialization option 
in LC.

Being binary, like the BSON used by MongoDB and others, it's best used 
machine-to-machine, not suitable for direct human editing using a text 
editor.

But being so with-the-grain it's easy to make viewers and editors for 
array data as you need them.

And if you anticipate doing a lot of manual editing you may find YAML 
very convenient to work with, designed like a lightweight outliner that 
just happens to work in any text editor.

Being optimized for LC, LSON is not only robust and convenient but also 
pretty fast and compact.  For example, numeric values are stored in 
their binary form, bypassing conversion to and from text.  With sparse 
delimiters and compact binary forms of numbers, in many cases LSON files 
will take less disk space and network transfer time than equivalent data 
in JSON (and far less than most XML).

Of course the main thing is to use whatever work for your project, and 
if you have JSON running well now there's probably no need to change 
anything.

For myself, I tend to pick among these formats like this:

YAML: When I anticipate needing to read/write them manually.

JSON: When I need to exchange data with other people's REST services,
       or when delivering data to a Web browser or other software
       that uses the JavaScript interpreter (like Node.js).

LSON: When the software reading and writing the data will be
       LiveCode, and not meeting the two previous criteria.

Lately most of my work has LC on the server feeding LC clients, so LSON 
has become my default go-to choice unless I have some specific need 
requiring something else.

I have one REST service for which I'll need to support both LC apps and 
Web browsers as clients, and I'm setting up the API so that the file 
extension in the request determines the output format, e.g.:

   get url "http://domain.com/cgi?query.json

...is parsed to run the output through an arrayToJson function before 
sending it back over the wire, while this:

   get url "http://domain.com/cgi?query.lson

...returns an encoded LiveCode array.

-- 
  Richard Gaskin
  Fourth World Systems
  Software Design and Development for the Desktop, Mobile, and the Web
  ____________________________________________________________________
  Ambassador at FourthWorld.com                http://www.FourthWorld.com