What is "Open Language"?

[Mark Waddingham]

On 2015-10-29 00:38, David Bovill wrote:
> From my point of view Open Language is the most important aspect of
> LiveCode. Without it on the timeline I would not be using LiveCode 
> today.
> I'd be working purely in NodeJS I suspect.

It is interesting that NodeJS and Open Language come up in the same 
post...

As it stands, LiveCode and NodeJS actually have a lot in common. Indeed, 
as long as you are careful about the syntax you use, and the operations 
you perform, they 'work' in the same way.

An event comes in, a message is dispatched to an appropriate object and 
then that object does a small amount of work, if it needs to do anything 
more heavy-weight it sets something in motion and ensures a callback is 
sent when it is done. The key thing here is 'minimal amount of work' it 
means that the approach is highly scalable in terms of concurrent 
requests being processed as they can all be interleaved very very 
efficiently.

In LiveCode, socket communication can be entirely callback-based in this 
model, as can network operations. However, the things that are missing 
at the moment are callback based variants of the database commands and 
other such things which might cause 'long pauses' whilst they are 
performed. (Really database operations and file operations need to be 
implemented in an event-driven way - so you start one off, and then get 
a callback when completed).

Now, one thing that would make such NodeJS type operations be even 
easier to use is if you *didn't* have to use callbacks. 'Straight line' 
code is much much easier to write. e.g.

on doMyStuff
   put url "..." into tStuff
   put revQueryDatabase(..., tStuff, ...) into tDbStuff
   put encodeDbStuffForResult(tDbStuff) into tResult
   return tResult
end doMyStuff

Is a lot easier to read and maintain than:

on doMyStuff
   load url "..." with message doMyStuff1
end doMyStuff

on doMyStuff1 tStuff
   revQueryDatabaseWithCallback ..., tStuff, ..., "doMyStuff2"
end doMyStuff1

on doMyStuff2 tDbStuff
   encodeDbStuffForResultWithCallback tDbStuff, "doMyStuff3"
end doMyStuff2

on doMyStuff3 tResult
   setResultOfRequest tResult
end doMyStuff3

(Note that here I'm imagining various things exist - 
revQueryDatabaseWithCallback does not currently!)

If one steps back, then essentially, the difference between the single 
handler version, and the callback handler version is that each operation 
in the single handler version is calling 'wait until ... with messages' 
(in some form or another). The problem with how LiveCode currently 
works, however, is the fact that script execution is heavily tied to the 
system stack (this is the stack which allows nested procedure calls - 
not a visual stack). This means that 'wait ... with messages' is 
recursive in LiveCode at the moment and not 'side-by-side'.

In the recursive model you have to use the callback-based approach 
because otherwise a single request might get stalled due to having to 
wait for another (unrelated) request to finish (as wait nests on the 
system stack - so all subsequent requests have to resolve but the 
initial one can).

Fixing the recursive model means that script execution has to be 
completely divorced from the system stack - essentially meaning that the 
engine can automatically turn the single handler model into a callback 
based model without the scripter being any the wiser.

How does this relate to Open Language? Well, one of the key pieces to 
implement Open Language is abstracting the way script is executed in the 
engine in LiveCode so that it no longer has any dependence on the system 
stack - scripts will be compiled to bytecode which is then executed. 
This extra level of abstraction gives a great many more choices about 
*how* scripts are run, and as such opens the door to resolving the 
'recursive wait' problem. This will open the door to bringing a 
simplicity of scripting with LiveCode to an environment where it perhaps 
wasn't quite so suited to before.

Now, of course, NodeJS does use the callback model (although the syntax 
of JS actually means it is perhaps a little less onerous than LiveCode, 
at least for one or two step processes) - so in terms of getting to 
parity with NodeJS the principal thing needed is callback variants of 
all operations which can block. However, given there are various things 
in the works in NodeJS to enable 'single handler' type models it does 
suggest that the callback model could be better.

Like most things, whilst Open Language might be an end goal in and of 
itself with all its benefits, the process of getting their open up a 
wealth of other opportunities too due to the improvements in the 
underlying technology that are made along the way.

Warmest Regards,

Mark.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps