Vector images?

[Mark Waddingham]

On 2015-10-20 21:35, Scott Rossi wrote:
> Mark, what you describe sounds great.  Even a subset of SVG 
> capabilities
> is very welcome.
> 
> But I have to ask, partly because I'm clueless when it comes to low 
> level
> programming, but also because I'm curious: does it make sense to have 
> two
> "realms" of graphics?  There are card based (native) graphics, and then
> there are the graphics inside widgets.  I recall reading that all of 
> the
> graphics rendering in LC was going to be moved over to the skia 
> graphics
> library.  Is this what enables the display of SVG in widgets or is
> graphics rendering in widgets based on something else?  What prevents 
> the
> display of SVG graphics outside of a widget?  As it stands, there are
> graphics capabilities within widgets that don't apply to native 
> graphics.
> Would it not make more sense to have a single universal approach to all
> graphics in LC?

There are a number of things to answer in your question, so I shall 
attempt to do so (apologies in advance, what follows is quite long - I 
thought it useful to provide a fair amount of background - answers to 
your direct questions are at the end!)

--

First and foremost, the most important thing to bear in mind is that 
widgets are a way to write engine controls in both a way which is an 
order of magnitude easier than doing so in C++ *and* in a much more 
accessible way (the number of people in the community at present who 
either have the necessary knowledge and/or interest in writing C++ is 
very very small). So, widgets are not special, they aren't doing 
anything the engine doesn't already have the facilities for - they just 
allow said facilities to be exposed at the LiveCode Script level in a 
much easier and quicker way than we could ever do in C++.

--

Moving on from that, the current 'stack' we have inside the engine for 
graphics is as follows:

Skia (entirely internal - not exposed at any level)
   => This is the 'rasterization' engine - it essentially converts 'paint 
this shape here' requests into actual pixels). It is a rather large and 
fiddly C++ class library which we have, over time, managed to persuade 
to work in the ways that we need it to.

LibGraphics (internal engine API - available for use by the engine)
   => This is a high-level, standard, PostScript-like C API allowing 
paths to be stroked and filled with various kinds of paints, 
transparency layers to be used and composited with bitmap effects, 
images to be rendered etc. It is very similar to CoreGraphics (but 
actually has a number of features beyond which CG offers). e.g.
      MCGContextCreate(-> myContext)
      MCGContextSetRGBAFillColor(myContext, 1, 0, 0, 0.5)
      MCGContextAddRectangle(myContext, [0, 0, 400, 400])
      MCGContextFill(myContext)

Above this we actually have two abstractions which sit side-by-side:

MCGraphicsContext (internal engine compatibility class - available for 
use by the 'classic' engine controls)
   => This is a wrapper around LibGraphics which emulates the old 
graphics abstraction in the engine and is used by all existing engine 
controls.

Canvas (LCB syntax extension - available for use by LCB)
   => This is a thin wrapper around LibGraphics providing 'nice' syntax 
to LCB to allow widgets to paint themselves:
     set the fill paint of this canvas to color [1, 0, 0, 0.5]
     fill rectangle [0, 0, 400, 400] on this canvas

So - all the existing ('classic') engine controls internally use an old 
abstraction called MCGraphicsContext, whilst all new 'engine' controls 
use Canvas (if written in LCB) or LibGraphics directly (if written in 
C++ - however, we do not intend to write any new engine controls in C++ 
as we have LCB - it would be a somewhat large 'waste of effort').

--

With the above in mind let's look at what SVG support actually entails. 
Firstly, SVG is 'just' an interchange format (although a very powerful 
and really quite complex one if implemented fully) - it is a high-level 
description of marks on a page. It is similar to the representational 
power of PostScript (Level 3 - which includes transparency) but is 
declarative (i.e. you describe explicitly what the elements are) rather 
than imperative (i.e. you run 'code' to generate the elements - 
PostScript is a language). SVG is actually (like all W3C standards) 
abstract - it is a well-defined data-structure which has a reference 
'encoding' as XML. So an SVG rendering 'stack' notionally looks like 
this:
     XML Importer - takes SVG XML and converts it to an XML document that 
can be manipulated)
     SVG-XML Processor - processes the XML document and produces the SVG 
data structure)
     SVG Renderer - takes the SVG data structure and turns it into actual 
graphics operations)
     Graphics Library - needs to provide the operations needed to render 
SVG primitives

'Unfortunately', Skia does not support SVG rendering directly - it was 
going to once it seems, however that aspect was abandoned long ago as 
not being worthwhile. I'm guessing this is because full support of SVG 
is far outside the realms of a (low-level) graphics (rasterization / 
abstraction) library. Instead Skia provides the underlying services for 
rasterizing SVG (e.g. the ability to render paths with arbitrary 
transforms with various different kinds of paint, process things with 
various kinds of filter etc.).

Now, obviously we only use Skia at a very low-level - it is strictly 
'hidden' behind our well-defined LibGraphics API (so that, if necessary, 
we can change the rasterization engine at some point, or abstract it to 
do various other 'neat' things - like 'export snapshot as SVG'). So, the 
first part of the work I've done is to add an MCGSvgRef abstraction to 
libgraphics. This neatly wraps up the top three things in the above list 
- it loads and parses an SVG XML document and then converts it to an 
internal form which can be rendered (note that the entire focus here is 
rendering - not editing or introspection on the SVG Document). There is 
then a simple call to render such a thing (which basically iterates over 
the internal form and calls appropriate LibGraphics APIs to do so). (I 
should point out that I didn't write the main piece of code which does 
the Importing / Processing - that would be 
https://github.com/memononen/nanosvg - I wrapped that up in a nice 
package and hooked it up to LibGraphics). The C API is actually this 
(for those that are interested):

bool MCGSvgCreate(const void *p_data, size_t p_data_size, MCGSvgRef& 
r_svg)
MCGSvgRef MCGSvgRetain(MCGSvgRef self)
void MCGSvgRelease(MCGSvgRef self)
bool MCGSvgIsValid(MCGSvgRef self)
MCGRectangle MCGSvgGetViewBox(MCGSvgRef self)
MCGRectangle MCGSvgGetBoundingBox(MCGSvgRef self)
void MCGSvgRender(MCGSvgRef self, MCGContextRef target)

With this nice simple abstraction in place (which is now generally 
available as part of the API exposed by LibGraphics to anything in the 
engine that might want to use it), I then wrapped it in LCB syntax for 
use by widgets. Indeed, it did not need very much syntax:

svg from file <filename>
svg from resource file <filename>
svg from string <xmlstring>
the bounding box of <svg object>
the viewing box of <svg object>
draw [ from <srcRect> of ] <svg object> into <dstRect> of <canvas>

So, at this point, any part of the engine can use the C-based API 
mentioned above, and any LCB code can use the 'nice' LCB syntax above.

--

The final piece to the current piece of work is a very simple (it is 
around 150 lines of LCB code, including GPL header comments) 'SVG View' 
widget which allows you to give it SVG as an XML string, and then tell 
it what portion to render into the widgets rect. (Here is the actual 
source - 
https://github.com/runrevmark/livecode/blob/lcb-canvas_svg/extensions/widgets/svgview/svgview.lcb)

--

Now with that all out of the way, let me see if I can answer your direct 
questions...

Q: does it make sense to have two "realms" of graphics? There are card 
based (native) graphics, and then there are the graphics inside widgets.
A: There aren't 'two realms of graphics' - what you are seeing is the 
graphics 'stack' at different levels. By 'card based graphics' I presume 
you mean the 'graphic' object. This is a 'widget' which allows you to 
display geometric shapes on a card. It sits on MCGraphicsContext 
internally which sits on LibGraphics. Similarly, any widget sits on 
Canvas syntax which sits on LibGraphics. Due to the graphics object 
current implementation atop MCGraphicsContext it has a number of flaws 
which are not fixable without (essentially) rewriting it...

Q: I recall reading that all of the graphics rendering in LC was going 
to be moved over to the skia graphics library. Is this what enables the 
display of SVG in widgets or is graphics rendering in widgets based on 
something else?
A: All graphics rendering in LC has been using Skia since around 6.5. 
Existing C++ engine controls do so through the MCGraphicsContext wrapper 
around LibGraphics (which actually sits on Skia). SVG rendering support 
has been added at the LibGraphics level (using a third-party SVG parser 
/ processor), which has then been exposed to the LCB Canvas syntax for 
widgets to use.

Q: What prevents the display of SVG graphics outside of a widget?
A: Nothing - the SVG support is available at the LibGraphics level and 
so any part of the (C++) engine could use it. Really this question 
exposes a slight misunderstanding about widgets. Widgets are no 
different from the other engine controls - they are just written in LCB 
rather than C++ which means they are a great deal easier and faster to 
write *and* most importantly, a great many more people will be able to 
write them. (Thus providing, over time, a great many more 'engine' 
features for everybody).

Q: As it stands, there are graphics capabilities within widgets that 
don't apply to native graphics. Would it not make more sense to have a 
single universal approach to all graphics in LC?
A: Internally and from an implementation perspective (whether it be 
available at the C++ or LCB level) there is a universal approach to 
graphics in LiveCode. However, I think what you are asking here is 'why 
does there appear to be more graphics facilities available at the Canvas 
API level than we can see through the current graphic object'... The 
answer here is simple - because we have not yet rewritten the graphic 
object. The current behavior of the graphic object is such that trying 
to 'move it forward' into the modern era is, essentially, pointless - it 
would have to be done with a big hefty flag saying 'use new behavior' 
(as otherwise all existing stacks using it would not work the same as 
they did before), and that is equivalent to just rewriting the object 
from scratch. This is the much prophesized 'shape' object. Which we will 
write in LCB (and, indeed, will actually just be quite a simple 
'wrapper' around the Canvas syntax in the first instance at least).

Warmest Regards,

Mark.


-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps