OpenLanguage: abstract syntax trees

[Mark Waddingham]

On 2018-04-19 09:57, David Bovill via use-livecode wrote:
> So let's ask some dumb questions:
> 
>    1. Is there an abstract syntax tree (AST) for the Livecode language?

Yes - the AST is defined by the set of syntax classes in the engine 
source-code (such as MCAnswer, MCMinus, MCProperty).

>    2. Where can I buy it?

You don't need to - all the source-code is at 
https://github.com/livecode/livecode.git

>    3. How hard would it be to create one?

Well, one already exists so there is no real need to 'create' one 
per-se, although whether its current form is suitable for your needs is 
another matter - I suspect you aren't asking the right question :)

> 
> Google told me that I should use Coco, to generate an AST from the C++
> code. But i don't trust Google, I much prefer Siri due to her business
> model. My hope is that Mark Waddingham will explain this properly. I 
> trust
> Mark.

CoCo is a compiler front-end generator. It allows you to express a 
tokenizer as a set of regular expressions, and a parser as a formal 
grammar conforming to LL(k)ish 
(https://en.wikipedia.org/wiki/LL_grammar). CoCo then generates 
something which given a source-text will give you a syntax tree of some 
kind.

> *Open Language*
> My understanding is that we are steadily, if not slower than hoped, 
> moving
> towards open language in Livecode. It's something I need, hence my 
> interest
> in DSL's, AST's and compilers. I'd like to know more.

Yes we are still moving towards it, but it is taking a lot longer than 
expected (although lc-compile, the LCB compiler, does have a non-dynamic 
version of Open Language to a certain extent which, unfortunately, isn't 
suitable for LCS in its full gory detail).

> At the moment I believe from previous posts, it is possible to create 
> DSL's
> in Livecode, but you have to compile from source. I'd like to have a go 
> at
> that if possible, as it may well be a shorter path to creating my DSL, 
> than
> trying to get Siri to teach me about abstract syntax trees.

Without knowing any details of the new project, or what you would like 
to be able to do it is hard for me to advise here.

Certainly augmenting engine syntax is possible by writing C++ and 
building from source - although the difficult in doing that is entirely 
related to what syntax you'd like to augment / extend / change. That 
wouldn't really create a DSL per-se, but an extended version of LiveCode 
Script (which may be what you want, or maybe not).

Indeed, it is entirely possible that the DSL you want to create doesn't 
require fettling with the engine at all - LCS is a really good language 
for writing DSLs as it stands, as well as interpreters and compilers. 
For example, Ali's tutorial system is a simple DSL, using a parser and 
interpreter both written in LCS 
(https://github.com/livecode/livecode-ide/tree/develop/Toolset/palettes/tutorial).

> Mark any chance of sending the gift of your knowledge our way?

I could ramble on at length ( not that I'm prone to that, of course ;) ) 
about designing programming languages, interpreters, compilers, parsers, 
tokenisers, name analysis, abstract syntax trees, concrete syntax trees, 
code generation and a great many other things...

However, I suspect something more focused would be more helpful and for 
that I really need a greater idea of what you are wanting to achieve - 
in particular, what is the DSL you are wanting to create, how do you 
want it to be used, and what is it intended to do?

Warmest Regards,

Mark.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps