What is "Open Language"?

[Mark Waddingham]

On 2015-10-24 21:12, Richard Gaskin wrote:

> My favorite example is the "export snapshot" command - even with the
> Dictionary guidance we all need to turn to for that no matter how many
> times we've used it, it's so complex with such a combinatorial
> explosion of options that I defy anyone to make a readable BNF for it.
> :)

The 'export snapshot' command is rather unfortunate. It was augmented by 
me to add the ability to snapshot an object directly (rather than the 
screen) and because (to be blunt) the actual code for the syntax of the 
export command was always a complete 'botched' job I did the best that I 
was able to do.

Indeed, I'd defy anyone to make truly accurate BNF for *any* of the 
language's current syntax - this is actually a tremendous problem which 
limits a great deal. The parser is very poorly implemented in this 
regard. It was never done with any amount of rigour, nor following any 
standard structured techniques for such things (it dates back from long 
before my tenure). Indeed, having spent many years studying programming 
language design and implementation before I joined RunRev (as it was 
then), when I actually saw how it was implemented to say I was 
'surprised' would be an understatement.

(It is extremely important to understand that perturbing the parsing 
code in any way runs the risk of breaking existing scripts... Precisely 
because it was never written to a high enough specification standard).

Now, that being said, it obviously does work - and given the amount of 
code out there which is run through its internals the outward effects of 
however it is implemented internally is not necessarily affected. What 
is hugely affected though are things which are incredibly important if 
you want to do describe, extend or do any 'clever' processing of the 
language.

The problem is that because the parser is so lax, and almost impossible 
to pin down to a tight specification of what it recognizes, the actual 
definition of what LiveCode Script currently is syntax-wise, is entirely 
defined by the implementation of the parser.

After the refactor work we did for 7, however, we can actually finally 
do something to clean this up. We have 'unhooked' the execution of 
parsed commands from the representation of the parsed commands in 
memory, and thus this means that it will make it possible to implement a 
tool which maps an existing parsed script to a cleaner syntax.

Before anyone gets concerned, I'm not proposing anything drastic here at 
all. Each piece of syntax the current engine understands will have a 
representative form chosen which will then be used for the 'unparsing'. 
The original parser would be used to compile each line, and then the 
result put through a reverse operation to generate string matching a 
normalized form in each case. For example, one of my favourite examples 
of the unwieldy parser is 'accept connections' as you can do this:

    accept accept accept accept 1000 with empty "callbackMessage"

The (normalized) representative form would be:

    accept connections on port 1000 with message "callbackMessage"

If the syntax of the language can be pinned down to a simple variant of 
Backus-Naur Form (BNF) then a great deal of things become a great deal 
easier.

> For example, if "export snapshot" were an R command it might look
> something like:
...
> ...gives you a gorgeous scatter plot with useful x- and y-axis labels
> and well-placed tick marks, without having to specify anything; you
> can specify as much or as little as you like with most commands and
> expect a useful result.

That's great. I'm glad R does things like that - I'm sure it has 
developed its ideas and forms as appropriate to its explicit very 
restricted domain (and a great great deal more restricted than  
LiveCode).

> I'm not proposing LiveCode switch to name-value pair arguments, but in
> some cases it would be a nice option, "export snapshot" being among
> them.

Well it is a possibility - but, like adding 'dot' notation - I have to 
ask is that the right thing for LiveCode (xTalks)... Surely it would be 
better to work out how the English-like nature is best preserved, and 
implement a system which means it can be done easily...

Oh wait - that's the point of Open Language :)

Warmest Regards,

Mark.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps