Monte Goulding's day off? ; -) mergJSON - Documented Bug or Feature?

[Lagi Pittas]

HI Mark & Monte,

"I do wonder here whether the problem is that from LiveCode Script's point
of view (and Lagi's use-case, probably others too) that it would be
preferable that all non-object/array/string values in the JSON are returned
as they are in the input file;  ......."

The above is exactly what I have been alluding to.

Don't get me wrong I get a bee in the bonnet - maybe not a bee but at least
get irritated --  when something that I think is near damn perfect and
works - as the MERGJSON did - does something I believe is not logical or
could be so much easier/better/faster (stronger and jump tall buildings?)
with just a little change.

Here's the deal - if I read in a CSV file and change 1 character and write
it all out again it makes no sense to have anything except that 1 character
change whether double float or whatever - if that is what the JSON rfc says
  should happen  (and we can still debate that) then I'm sorry it is wrong
plain and simple. I will defer to you greater understanding of the
minutiae but i'm looking at it in a Ruby type fashion - "The Least
Surprise".

Anyway I think from the last response you have understood where I was
coming from so I am sure it will be sorted.

Oh and Monte - the reason for the subject line? At least it would get read
.... ;-)

Regards Lagi


<ASIDE>

"Lucky I didn't tell them about the dirty knife".





On 30 June 2016 at 10:16, Mark Waddingham <mark at livecode.com> wrote:

> On 2016-06-30 00:06, Monte Goulding wrote:
>
>> I’m sorry mergJSON isn’t working out for you. It sounds to me like you
>> actually want to represent your numbers as strings in the JSON. In
>> which case mergJSON does have the facility to force things that look
>> like numbers to be strings and things that look like sequentially
>> indexed arrays to be unordered objects.
>>
>
> This relates to the generation of JSON, I believe?
>
> I think the issue here is the reading of the JSON.
>
> LiveCode will format numbers using the current numberFormat when the
>> user wants to use them as a string. This would probably be a
>> reasonable change to the external too. mergJSON is open source so if
>> anyone is so inclined they could use EvalExpr to get the numberFormat
>> then do something like what the engine does in MCU_r8tos to format the
>> string. The repository is at
>> https://github.com/montegoulding/mergJson. We might want an extra
>> boolean parameter governing whether to apply the numberFormat also I
>> think.
>>
>
> I do wonder here whether the problem is that from LiveCode Script's point
> of view (and Lagi's use-case, probably others too) that it would be
> preferable that all non-object/array/string values in the JSON are returned
> as they are in the input file; string values being unescaped appropriately
> (the quotes removed, obviously). This means that the LiveCode engine would
> then do the conversions as appropriate, during use.
>
> In particular, this would mean the original representation of numbers as
> they are in the JSON would be preserved - i.e. you would get 1.40 if the
> value in the JSON was 1.40 - not the result of going string -> double ->
> string again.
>
> Unfortunately, from what I can see in the jannson source (the library used
> my mergJSON to read the JSON), there is no mode that allows this.
>
> As LiveCode Script can't distinguish between
> numbers/strings/null/true/false - all are represented by strings, this
> would actually be better than the current situation for many use-cases.
>
> However, it is important to consider the fact that there are use-cases
> where you *do* need to distinguish between a string "1.40" in the JSON
> input and a number 1.40; as well as a string "true" and the boolean value
> true in the JSON input. This is a similar problem to supporting 'NULL' in
> database columns - LiveCode Script doesn't distinguish between the empty
> string "" and no value at all (NULL) - so you have to use a sideline call
> to determine if a database field is actually NULL or not.
>
> Prior to LiveCode 7, all values were either strings or arrays - however,
> now they can actually be nothing, booleans, strings, numbers, binary
> strings or arrays and there are operators to determine what a value's type
> 'really' is the 'is strictly' operators. So you can do things like:
>
>     put "1.40" into tNumString -- tNumString is strictly a string
>     put tNumString + 1.6 into tNumber -- tNumber is strictly a number
>
> These could provide a basis for providing a JSON importer which allows
> both modes of use:
>
>   1) The 'simpler' everything is a string or an array view.
>
>   2) The 'faithful' things have a specific type view, where you can use
> the 'is strictly' operators to determine the *actual* type of the value
> which was specified in the JSON input. e.g.
>       {
>          "foo" : true
>       }
>      Would return an array with key "foo" mapping to 'string' "true"; but
> if you test the value with 'is strictly a boolean' it would return true; if
> you test with 'is strictly a string' it would return false.
>
> There's still a couple of caveats here...
>
> LiveCode Script (as yet) does not have 'proper lists' - at the moment
> 'arrays' are used to represent both JSON objects and JSON arrays (JSON
> arrays are sequences of values - lists in our parlance) - so you cannot
> distinguish between something which is an object in the JSON input and
> something which is an array in the JSON input.
>
> I'm not sure yet how we'd make it possible to distinguish between the
> number 1.40 and the string "1.40" in the JSON input but still make it so
> that you 'see' what the precise input (1.40, in this case) was when the
> number is converted to a string. Maybe one option would be to have a new
> internal datatype StringyNumber which would be a String, but act like a
> Number - basically a pair (string, number) with the invariant that
> stringtonum(string) == number (but not the other way around - as that's
> ambiguous). Such a beast would return 'false' for is 'strictly a string',
> but 'true' for is 'strictly a number'. Of course, the problem then is that
> you get an inconsistency with regards things such as 'the numberFormat'.
> Whenever the engine formats a number as a string, it uses the numberFormat
> to work out how to generate the string representation of the number. With
> StringyNumbers, though you get an inconsistency:
>
>     put 3.14159 into tStringyNumber
>     set the numberFormat to "#.##"
>     put "Number: " & tStringyNumber
>     -- as tStringNumber is strictly a number you'd expect it to obey
> numberFormat and so the output would be:
>     --    Number: 3.14
>     -- However, the point of a stringy-number is to preserve the original
> string rep, so with the currently proposed logic
>     -- you would get:
>     --    Number: 3.14159
>
> This means the 'naive' approach suggested above is perhaps not quite right
> as it makes the number/string 'fragile' in the sense that you can get
> different outputs depending on what the inputs were (and have no way to
> detect that difference will occur). Therefore, I wonder if we can make the
> numberFormat logic 'work' on strings, and not numeric representations...
> i.e. The operation ConvertToString(StringyNumber -> String), should use the
> string part of the StringyNumber together with the numberFormat part to
> generate the output String - essentially if the numberFormat *requires*
> more trailing decimal places than the string part of the stringy-number
> has, then it should extend the string with those zeros to match; if the
> numberFormat *forces* less decimal places than the string part of the
> stringy-number has then it should format from the number part of the
> stringy-number, ignoring the string part altogether (the other option would
> be to 'round' the string directly, I think these two these would be
> equivalent - but a little bit of maths is needed to see which is the most
> appropriate to minimise error).
>
> Anyway, upshot is, mergJSON uses a library which perhaps does not do
> things 'quite the best way' for integration with a stringy language like
> LiveCode Script (hence the contention Lagi is having); the recent changes
> to LiveCode (since 7) could mean we can have the best of both worlds in the
> future, although there is a bit of core engine work to do to make that
> possible.
>
> Warmest Regards,
>
> Mark.
>
> --
> Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
> LiveCode: Everyone can create apps
>
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your
> subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode