Documentation on Accelerated Rendering

[Mark Waddingham]

On 2017-04-30 23:52, J. Landman Gay via use-livecode wrote:
> I have been wanting to know that for years. I have the general idea
> but I want to know exactly what each command or property does. How
> does layerMode interact with rendering and when should I set it? (I
> used it incorrectly and submitted a spurious bug report, but I'm still
> not clear on why I was wrong.)

This is perhaps something we should have more clearly documented at the 
time,
it has been a while since I last looked at it...

Your description is generally correct, but I'll see if I can elaborate 
with
more technical details.

First of all, just a reminder about how the engine handles screen 
updates...

The engine checks for the need to update the screen after every command 
has executed, or after 'unlock screen' if the screen has been locked and 
the unlock causes the lock count to reach zero.

During execution of a single command (when the screen is not locked), or 
the execution of all commands if under lock screen, any required visual 
changes to the object are accumulated in the stack's 'dirty region'. 
This is a collection of rectangles (on the stack) which need updated.

The acceleratedRendering property changes the way the engine renders the 
stack when changes are applied (either after each command, or after an 
unlock screen which resets the lock count to zero).

When it is false, the engine creates an offscreen buffer big enough to 
cover the dirty region, uses the dirty region as clip, then iterates 
through all objects on the card back to front, rendering each in turn 
into the single offscreen buffer. When that is complete, the engine 
copies the offscreen buffer to the screen.

When acceleratedRendering is true, however, this changes. Accelerated 
rendering mode views the tree of top-level objects as a flat sequence of 
layers, ordered from back to front. Each layer can have one of three 
layerModes:
   - static
   - dynamic
   - scrolling

The static mode indicates to the engine that that layer is not going to 
move, so once rendered once (in theory) it and any thing it covers won't 
need to be re-rendered.

The dynamic mode indicates to the engine that the layer is going to 
move, so ideally should be rendered independently of everything else.

The scrolling mode is the same as dynamic *except* that it indicates 
that the layer is just a view (clipped rectangle) into a larger area - 
e.g. what you can see in a field currently, and what the field would 
look like if rendered on an infinite screen with no clipping.

When there are multiple static mode objects next to each other in the 
list, they are all elided into a single layer. e.g.

    Graphic - static (bottom object)
    Image - static
    Image - static
    Button - dynamic
    Image - static
    Field - static (top object)

This would generate three separately rendered layers - the bottom one 
being the first three controls, the middle one being just the button, 
the top on being the last two controls.

The key thing here is that these layers can just be composited - i.e. 
they are existing bitmaps, which are blended together into the screen 
buffer on demand - this is something GPUs are very very very good at. In 
particular, when you have moving (dynamic) layers, moving those 
layers/objects doesn't require re-rendering anything *just* 
re-compositing to the screen with the layers in a different place. This 
is the source of the speed improvement you can see with 
acceleratedRendering (e.g. you can have, say, 100 objects moving about 
smoothly whereas you'd only manage maybe 10 before).

This is not the whole story, as acceleratedRendering mode makes one 
other 'optimization' - it doesn't actually use object-sized buffers for 
any layer. Instead, it splits up each (merged) layer into tiles with 
size the compositorTileSize. So the compositing step is actually 
blending together lots of small bitmaps, rather than a collection of 
large bitmaps - this makes no real difference to performance (as it is 
number of pixels blended which is the key factor, not number of bitmaps) 
but does to updating things when any object changes as only those tiles 
which intersect with the dirty region need to be considered... 
Furthermore, in acceleratedRendering mode, the 'virtual layers' (which 
come about through merging together adjacent statics, but leaving 
dynamic and scrolling ones alone) each have their own dirty region. This 
means, much less re-rendering needs to take place (in any one case) 
*and* much less data transfer from memory -> compositor's tile storage / 
format need to take place.

In terms of the compositor properties:

   - the compositorType determines what kind of technology the blending 
step uses.
     There is a software mode, which works on all platforms and just uses 
the
     engine's not-too-shabby compositing code.
     There is a coregraphics mode, which works on Mac/iOS, which uses 
CoreGraphics
     (which gives an advantage because CoreGraphics images can be blitted 
direct
      to the window buffer)
     There is OpenGL mode, which works on Android/iOS, this uses 
tile-sized
     textures on the GPU, and the GPU to do all the blending.

   - the compositorTileSize determines the size of tile to use (which 
must be
     a power-of-two up to 256 pixels).

   - the compositorCacheLimit determines the maximum amount of memory to 
use
     for tiles.

The third point here needs a little bit of explanation. As 
acceleratedRendering mode splits things up into tiles, it can leave 
rendering any tile which is not
visible until it is actually visible. For example, if you have a large 
opaque dynamic layer over a stack of static layers, chances are that you 
will never see most of the static layers underneath, so the tiles for 
the (virtual) static layer will never need to be rendered, or stored - 
so it doesn't. This minimizes the number of tiles/textures/images which 
are created, and minimizes rendering time for any one update.

Now to answer you direct questions:

> - What does layerMode tell the engine, and when is it read? I think it
> tells the engine to create a buffer for the object. I had read a long
> time ago that it would help to reset the layermode to static if the
> object was done moving, but there should have been a caveat: don't
> reset if the object will move again later, because in that case you
> want to preserve the existing buffer. That seems to imply that
> resetting layermode to static will dump the buffer. Is that right?

When you change the layerMode it will cause a recalculation of layers 
around it (potentially) when the current dirty list (collection of 
per-layer dirty regions) is processed. If a layer is not going to move 
for quite a while, then you it might be reasonable to set it to static, 
as long as you can anticipate when it might move so you can set it to 
dynamic again (setting to dynamic and then moving immediately is likely 
to cause a stutter).

> - When set to true, what does acceleratedRendering actually do? Is it
> effectively a command to scan every object on the card and create
> buffers? Does it scan more than just the current card?

It changes the approach to rendering as described above. The 
accelRendering buffers, tiles and such are keyed to the stack, but 
flushed whenever there is a card change - so essentially it is card 
local (but you only ever have one accelRendering context per stack).

> - Is it okay to turn it on and off as needed? Are there disadvantages
> to that, assuming the objects won't be moving any more? Can I turn it
> on at the beginning of a handler and turn it off at the end if those
> objects won't be moving any more?

This would be ill-advised unless all movement you want is happening 
within that handler. Turning acceleratedRendering on will mean the 
engine has to re-render all visible tiles in all the virtual layers; 
turning it off again deletes the accelRendering context for the stack, 
dumps all its tiles and then re-renders the stack using the normal 
painter's algorithm into offscreen buffer approach.

> - When I turn it off, does it dump all the buffers? I think it does.

Yes.

> - What does it do between card changes? Does it keep old buffers until
> the cache limit is hit?

No - the buffer is flushed whenever the card changes.

Hope this helps!

Warmest Regards,

Mark.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps