Wait, the problem, and why it is important to solve

[Mark Waddingham]

So LiveCode has long has this feature called 'wait' - it is one of those 
seemingly innocuous things which, from the surface seems simple (it 
allows you to wait for something - it's great when syntax is aptly 
named!) however it is perhaps one of the deepest language features we 
have.

I think there is some confusion (or puzzlement) as to why I am a little 
hung up on it because, on the whole, the 'wait' command is not actually 
seen in script that much. Certainly it is part of some people's coding 
style (the one thing I love about LiveCode is that it promotes a very 
pluralistic style to solving programming problems), but on the whole 
many of us rarely if ever use it. Well, we *think* we don't use it.

The thing is that 'wait' is such an important feature to LiveCode that 
many pieces of syntax use it without you knowing! The pure mathematician 
in my likes classifying and counting things (hey its what we do!), so 
here is a reasonably complete list of the places the engine uses it 
(this list is alphabetic by source-file in the engine order):

    - it is used to provide the flash of the menu when a menu accelerator 
key is pressed

    - it is used to wait for uses of shell() to send back any data so it 
can be accumulated for return

    - it is used to wait for a return value from sending an AppleEvent on 
Mac (send and request)

    - *it is used to implement 'wait for/until/while ...' (kinda obvious, 
but hey, I was trying to be semi-complete ;))*

    - it is used to implement 'read ... with timeout' forms

    - it is used to wait for process death when you do 'kill'

    - *it is used to wait between beeps when you do 'beep <n>'*

    - *it is used to moderate the speed of 'drag from x to y'*

    - *it is used to moderate the speed of 'type STRING '*

    - *it is used to moderate the time between mouseDown and mouseUp when 
doing 'click at'*

    - *it is used to wait whilst a non without-waiting form of 'move' is 
invoked*

    - it is used in the mobile browser control to wait for JavaScript 
evaluation to finish

    - it is used to wait for mobile calendar UI modal panes to return

    - it is used to wait for mobile photo taking panes to return

    - it is used to wait for mobile contact UI modal panes to return

    - it is used to wait for mobile datetime/option picker modal panes to 
return

    - it is used to wait for mobile compose mail UI modal panes to return

    - it is used to wait for mobile media picker UI modal panes to return

    - it is used to wait for mobile fullscreen videos to finish

    - it is used to wait for mobile text message UI modal panes to return

    - it is used to wait inbetween vibrations when you use 'iphoneVibrate 
<n>'

    - *it is used to wait between frames for visual effects / 
transitions*

    - it is used in all forms of blocking socket commands (open, read, 
write etc.)

    - it is used to wait for blocking DNS resolution to finish*

    - *it is used to wait for 'modal <stack>' to finish (by closing the 
stack)*

    - *it is used (in libURL) to wait for evaluation of 'url <url>' to 
finish*

    - *it is used (in libURL) to wait for 'post to url <url>' to finish*

    - *it is used to wait for 'popup widget' to finish*

    - it is used by various other things in externals, and widgets and 
libraries to implement things which need to wait for something or some 
time in order to have something to return

Phew! The ones surrounded by '*' are ones which would make some sense to 
have working in HTML5. Almost all of these can be rendered in a form 
without 'wait', they just end up being *slightly* less useful or easy to 
use (e.g. get url -> load url, always use 'move without waiting', etc.).

So, first thing to point out, even without 'wait', if we manage to 
implement all the existing things which make sense in the HTML5 engine, 
'greenfield' (new projects) will work fine (as you avoid the things 
which don't work in HTML5), and a reasonable number of existing projects 
(particularly ones which are already written to use 'load url' rather 
than 'get url') will also work. However, there will still be a 
substantial number of existing projects which will not, without 
significant code changes.

One of the things we pride ourselves on is that (on the whole) an app 
written on one platform will work, largely unchanged on all the others 
(particularly if platform-specific features are avoided). However, we 
can't really keep that mantra, if something such as 'wait' is abandoned 
because it is deemed 'too hard to implement'. It means for any project 
targetting HTML5, you have to write in a different style, and avoid a 
reasonable number of features that many use day-to-day.

Now, if these features were 'just' things like 'drag' or 'click' or pick 
any specific (non-general language feature) that we have things wouldn't 
be too bad. However, 'wait' is a different kettle of fish altogether.

Wait is essentially LiveCode's implementation of 'async' - this is 
something that C# has (which is a much newer language than LiveCode - 
and its HyperTalk heritage); it is something JavaScript is in the 
process of adding. The simple reason is that event-driven coding is so 
ubiquitous these days (we've known its important for a long time) that 
other languages are gradually catching up to the need to have some way 
to do it easily.

Of course, async (i.e. wait) in LiveCode is not perfect - it is 
recursive, and not side-by-side (or 'round-robin' would be another way 
to put it) - and it is a little bit specific. However, on the whole, the 
fact that it exists at all *and* the engine uses it in various commands 
*does* simplify things.

So, 'wait' is important. Indeed, one of my hopes is that through getting 
wait to work in HTML5, we will also be able to *then* improve wait to 
bring it on par with other languages much more recent 'async' 
implementations. Indeed, my hope is that we can get to a point that code 
which needs to be written in a threaded/callback style now in LiveCode 
could actually be written without callbacks - this is the goal of the 
additions to JavaScript (due to node.js and its pure event-driven 
nature) - and if they can do it without having had a 'wait' 
implementation to build on, why can't we?

Anyway, the 'wait' beast in HTML5 has almost been slain (at least in 
thought). I think we almost have a solution which will require modest 
C++ code changes (which are doable without huge churn); and the rest can 
be done as a process that transforms the scripts we write. It might be 
upon further analysis, that another angle of attack actually makes more 
sense - compiling the current syntax to a bytecode, and using a VM which 
understands how to wait to execute it. To get performant code out, the 
asyncify algorithm / process would still be needed at some point; so it 
is merely a matter of ordering. At the moment, I *think* asyncifying at 
the script level will take less time to engineer and thus gets us to a 
fully compatible HTML5 engine quicker; however, we shall see what the 
next few months bring.

Warmest Regards,

Mark.

P.S. One other possibility I've toyed with is doing LCS->BYTECODE, then 
BYTECODE->ASYNCIFIED_JAVASCRIPT. The latter would be particularly easy 
if targetting browsers which have already implemented the new async 
JavaScript features. Since it looks like the HTML5 engine will only 
become truly widely usable when we move to WASM, this might well be a 
much more maintainable, and relatively quicker option.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps