Business Application Framework

[Mark Waddingham]

On 2015-08-12 23:35, Peter TB Brett wrote:
> On 2015-08-12 22:52, Richard Gaskin wrote:
> 
>> Now that we're talking about a much broader scope, and especially
>> given the central role of VCS in fostering healthy open source work,
>> my opinion is now more open than before, and somewhat undecided.
>> 
>> If it turns out that we've had a great open source option the whole
>> time and just never realized it, the situation is somewhat mitigated.

Unfortunately this is not entirely true - lcVCS and similar approaches 
unfortunately suffer some serious flaws in day-to-day use.

>> I don't know if lcVCS is available under GPL-compatible license, and
>> if so that would seem a good option.   But then again, if it's a good
>> option why would LiveCode Ltd undertake the non-trivial expense of
>> writing a completely different tool?

The features required by lcVCS (i.e. exporting a stack in an array, and 
then storing on disk as an exploded format) seemed like very sensible 
engine features. i.e. They add something to the language beyond just 
giving you something which can be checked into VCS such as Git. I've 
always wanted to generalize the stackfile import/export process in the 
engine for a variety of reasons, and this was a good first step towards 
doing so. (Note that lcVCS could have been rehosted to use the 
import/export features that we were adding in the engine - it is only 
one piece of the VCS puzzle).

> It turned out to be impractical to do this any better than lcVCS does,
> and lcVCS is already free software that any of our users can use, so
> my project got shelved around Christmas 2014 [1].  If you want to see
> where I got to, go and look at:

There was actually two reasons the project was shelved (the features are 
still 'useful' and I'm sure we will return to them at some point - if 
nothing else to abstract stack loading / saving in a way which makes it 
easy to store stacks in a variety of ways).

The first reason was resources - we needed Peter's considerable skills 
elsewhere. In particular, helping out with LCB and the widget 
infrastructure... That project was seen as considerably more important 
to get done sooner as without the basics done and working, nobody could 
actually start writing widgets and such to validate the approach. So, 
Peter was moved to work on LCB and related aspects and as a result we 
got a version we could use considerably sooner than we would have 
otherwise.

Of course, one could ask why VCS was considered to be less important and 
thus mean this shelving was considered. This is reason two. The reality 
is that lcVCS and our similar solution do not meet the goals that are 
required of collaborative development using VCS. Sorry.

In order to do collaborative development using VCS it is not enough just 
to be able to put your files into git in a way where most merge 
conflicts can be mitigated (i.e. automatically resolved in a correct 
fashion). The format in which you see the files on git and, most 
importantly, as PRs have to be human-readable. This is where UUID based 
on-disk formats fail - given a PR and its patch it is exceptionally 
difficult to work out in which objects the changes are being made. Given 
that LiveCode allows (and indeeds encourages you!) to spread code out at 
various levels and in various objects - this is a killer. It means that, 
for all intents and purposes, doing code reviews on PRs that are 
submitted to GitHub is exceptionally difficult, if not impossible as you 
have to do significant amounts of clicking and prodding to find out 
which objects changes are being applied to.

This flaw was actually pointed out by an engineer working on a project 
where they had attempted to use lcVCS when he was reviewing our version 
of the on-disk format. As the number of people on a project grows, there 
becomes even greater need for code reviews and management of the flow of 
patches and thus visibility and introspection on the patches by a human 
becomes even more important. (By the way it isn't just code reviews 
where this visibility of changes is exceptionally important - its also 
important when looking back through history in the commit logs to try 
and find where issues are introduced and other quality related issues).

One of the goals of lcVCS-like solutions is to integrate with systems 
such as GitHub so that people can collaborate on LiveCode projects in 
the same way as they do for other languages, when you come to this point 
it suggests that either (1) there is a flaw in the idea of using a 
system such as GitHub as a way to VCS LiveCode stacks or (2) the current 
way LiveCode programs are written is hugely dVCS-averse (at least for 
modern software engineering workflows).

We considered many options here. There was a general distaste for all of 
them - one would be to require unique object tags for objects in a stack 
to ensure you *can* see what objects are being patched. However, this 
puts a huge constraint on moving existing projects into the system, and 
also a requirement that you code in a slightly different way (it also 
considerably increases the possibility of difficult to resolve merge 
cases in various places, unless you rethink the stack folder structure 
considerably and I'm not sure we actually came up with a reliable way). 
Another was to produce a web 'front' for GitHub that shows you these 
things in a more friendly way. This was considered to be contrary the 
original goals of the project - i.e. if we have to go to the effort of 
producing a web front end to GitHub so that people can use GitHub for 
LiveCode projects, then why are we trying to contort things into GitHub 
style code repos in the first place.

Perhaps someone, at some point, will have a bright idea about how to 
make general stackfiles work well collaboratively in GitHub and similar 
systems. That will mean that you can follow a standard, quality, 
engineering workflow for LiveCode projects without having to put 
constraints on the way you write LiveCode projects. However, with the 
lack of such a solution (and a lack of time to further investigate 
potential options - it is, it would seem, a very difficult problem), we 
turned the problem upside down. Rather than ask "how can we produce a 
system which allows dVCS on arbitrary LiveCode projects?", we asked 
"What constraints would need to be put on a LiveCode project to allow it 
to be used in dVCS in a way a group of engineers can work together on a 
project in a standard engineering workflow?".

Ultimately that is what produced the precursor to the BAF. This was a 
bespoke solution for a specific project, which I then took and 
generalized using the insight I had developed working with Peter to 
develop our variant of the lcVCS on-disk format).

Mark.

-- 
Mark Waddingham ~ mark at livecode.com ~ http://www.livecode.com/
LiveCode: Everyone can create apps