How We Can Have Nice Things

[Richard Gaskin]

Yesterday I wrote that I'd report back on my meeting with the core team 
regarding community contributions on binary LiveCode stack files.  Many 
good things came out of that and other recent meetings, but in the 
interest of time let me focus on that one task here:


Why It's Been Difficult To Have Nice Things
-------------------------------------------
To recap the story thus far, one of the things we enjoy about LiveCode 
is that it give us the freedom to make our own decisions about how (or 
even if) we separate code, data, and content.  LiveCode stack files are 
enormously flexible, and can be used for any of those three, or any 
combination of them, as we choose for the task at hand.

Part of what keeps us choosing LiveCode is that most alternatives treat 
UIs as an afterthought, usually completely separate from the core 
language and almost always in files separated from code.

This has resulted in a world where other people's version control 
systems only account for those common ways of working, unable to 
anticipate something as uniquely valuable as LiveCode.

With LiveCode being open source we need to bridge any gaps between the 
tools that are available and the work we need to do.



Most Things Can Be Made Very Nice Right Now
-------------------------------------------
Fortunately, as Monte pointed out this afternoon, most of what we need 
to do is well supported by Github already, allowing the core team and 
the community to collaborate on everything in the engine and the docs, 
and most of the libraries in the IDE (thanks for assembling those stats, 
Monte):
<http://lists.runrev.com/pipermail/use-livecode/2015-September/218670.html>

For the remainder, most of what we need to do has been well covered by 
what we affectionately call the FIX Protocol:  if a community member can 
fix an IDE bug and provide the fixed handler in the bug report, just 
changing the report's summary to be prefixed with "FIX:" allows it to be 
identified easily, and since the solution is already provided it can 
usually be incorporated quickly.  So far I believe 100% of FIX 
contributions submitted through the bug DB have been folded into the 
product (thanks to Mike Kerner, Mark Wieder, and others who've provided 
those).

Between these two mechanisms (Github and the "FIX" protocol) I agree 
with Monte's assessment that more than 90% of what comprises the 
LiveCode product is readily available for community contributions right now.



The Handful of Things That Can Be Made Nicer
---------------------------------------------
This still leaves us with a relatively small but sometimes very useful 
subset of enhancements not covered by either Github or simple FIXes in 
the bug DB.

The one that started this thread was Charles Warwick's generous 
contribution of a nice Script Editor enhancement to add filtering to the 
handler list.

The core team recognized the value in that right away, and added it to 
the Development branch for v8.0.

But since work on IDE UI elements must be done manually, given the long 
queue of other things we want them to do it isn't practical for them to 
do such manual work twice so it can also be available in v7.x.

These are fairly rare occurrences, but it would be nice if we had a way 
for the community to make such enhancements and have them available 
right in the product so everyone can enjoy them in later 7.x releases.

And that's where we come in:



How We Can Have Nice Things
---------------------------
As dev lead Mark Waddingham noted this morning, the main concern with 
accepting complex stack files like the Script Editor from outside 
sources is that the company has a responsibility to ensure fitness and 
security for all LiveCode users.   And being binary files, there's 
nothing in Github to support the key steps needed for all such 
contributions:

- IDENTIFY specific changes between a master stack file
   and a modified one.

- REVIEW those changes to ensure fitness, compatibility,
   and security.

- MERGE those changes into the master, if for some reason the
   master has been altered since the changes were submitted.

That last step, Merge, would be the most complicated, but thankfully it 
may not be needed at all if we used a sort of check-in/check-out 
procedure.  Rather than build some system for that just for the few 
cases it would be needed, since we're talking about an older branch than 
Development most of the time it's not undergoing internal changes 
anyway, so it should be easy to arrange to communicate when you want to 
change a file, and the team can simply do something else for a couple 
hours until you're done and send the modified version back to Github.

That leaves us to focus on the first two tasks: How do we identify 
specific changes between two stack files, and make it easy to review 
those changes?

If we can come up with a sufficiently useful tool to help with that, the 
team is quite willing to consider accepting some changes like Charles' 
in modified stack files.

I think it would be helpful to keep in mind that we don't want to go hog 
wild, rewriting tons of objects and overloading the team with these 
types of out-of-process change requests.  They do have a lot on their 
plates that their already committed to, esp. the remainder of the Road 
Map, so where we can let's help those current efforts and keep other 
additions to a minimum.

I also think it would be helpful to consider simply submitting a feature 
request to the bug DB whenever practical, since that allows the team to 
make an implementation in the Development branch which integrates well 
with other long-term goals.

But for things like Charles' handler filter, we can move forward if we 
can not only make a modified version of the last RC build of 7.1's 
revscripteditor.rev file, but first put together a tool to allow the 
team to clearly see where it's different.


Making Nice Things
------------------
FWIW, some time ago I make a simple tool that copies all object 
properties of a given stack file, including scripts, into an array, and 
does the same for a modified copy of that stack.  Comparing the keys and 
values in the array yields a list of differences, with a two-field 
display below the list to show what those values are.  For lengthy 
values like scripts it also includes a very rudimentary DIFF to quickly 
highlight differences.

That said, as it is the tool is close to useless for any but the 
simplest of stacks.  Adding a single object results in more than a 
hundred entries, since every property is a delta.  Comparing Charles' 
stack with the shipping one showed more than 5,400 differences.  No one 
wants to wade through that.

Before we consider whether it makes sense to use an array-based 
comparison method and clean up the display with a simpler view of 
deltas, I know some of you (esp Mark Wieder and Monte) have already come 
up with nifty tools to translate stack files to and from XML or other 
formats that may lend themselves to efficient review.

If we as a community want to see things like Charles' handler filter 
added to the pre-Development branch IDE, let's discuss what tools we may 
have available and how we can put together one good solution for 
efficient review of stack file changes.

Personally, I'm happy to contribute to this effort if shared under GPL 
or GPL-compatible license, and I think such a tool would be useful for a 
great many other things beyond the small subset of occasional IDE 
enhancements that don't fit into other workflows.

Anyone here interested in contributing to such a tool?

-- 
  Richard Gaskin
  LiveCode Community Manager
  richard at livecode.org