Understanding LiveCode Source [BOM Issue with livecodescript URLs]

[Mark Waddingham]

The best way is to ask 'us' (as in the engineers that have worked on the entirety of the LiveCode source base for many years). This isn't any different from any other open source project which has 1.5m lines of code (I don't think at least...).

We can certainly provide an insight at where to start looking in terms of working on most issues - although don't be surprised if some result in a littany of 'that would be nice but...' type response. (The engine, in particular, has yaks needing shaving all over - as the years go by they are getting increasingly sweaty and tetchy because of it).

It is easy to get disheartened (slightly?) when working on the engine itself - but please don't be. We (as the maintainers) of the source base will offer as much help as we can, but do have to balance that with everything else we do.

I'd like to think that we do better than the RTFS responses you see in many other projects - but it really does come down to that in many cases (as we have to do that ourselves!). However, we are are more than happy to dig out the 'lower hanging fruit' to help ease learning - you've already found one of those...

I think you've found (one of) the good spot(s) to start - handling BOMs needs to occur at the point the engine reads in the text of the stackfile script as data and then use it with an 'external representation' bool flag to the decode API call in libfoundation (MCStringCreate...). I think that flag is currently unimplemented (so a small yak shave but not in anyway huge).

You've already got a handle on the processes we use internally, and do expect every contributor to follow those processes, and sometimes we might sound blunt, but please always understand that in terms of ensuring what is done (in terms of mutation of the source - I use the term 'mutation' specifically as any change to a large existing, and existing entity can only be considered as such) is 'correct' (at least as far as the collective knowledge we hold is understood - and by 'we' I mean all that contribute).

In terms of best places to interact on these matters then here (the use-list) is probably not great - gitter is good for direct answers but our quality centre is better as it provides a much better permanent and grokable record (indeed - we try to file an issue for every change to make it traceable globally)... At least until a PR is submitted.

Also we *could* invite dedicated external contributors to our private Slack - not something we've done as yet but it's a very tangible possibility. (And this is not a matter of privacy - but of pragmatic information exchange - those that work for LiveCode directly have time constraints on responding to input - that's all).

So, anyway, perhaps a much longer response than you expected. However it was not aimed at you (Brian) specifically, but generally. We don't expect contribution from anyone, but are always incredibly happy and as supportive as we can be when it occurs.

LiveCode is a wealth of interesting problems, if nothing else, and you can learn a lot from it (as I have done in my 13 year tenure as 'chief engineer').

Warmest Regards,

Mark.

P.S. I made a comment recently about 'caution being one of the best tools to effect change' and it applies to any long standing entity - of which LiveCode is one. I completely reserve the right (as 'BDFL' which is perhaps my title in that space) to cessate anything which I do not think is right (usually by closing a PR ;)), but I have an open mind - 'trust' is an important factor here on a technical level. I limit myself on what I do to the engine (in particular!) because it is very hard to get it 'right' when taking into the account the breadth of applicability - there are huge constraints which only become articulable when faced with direct reasons for any change. i.e. Start small and work up - anything else will result in a *seemingly* dismissive response. (ie don't expect huge support from us for very broad changes without long running interaction and justification).

P.P.S. I've learnt the above from tangible and *very* direct cost. The only reason I make these kinds of statements is because I care so much about what we are trying to be and do.

Sent from my iPhone

> On 5 Aug 2017, at 22:12, Brian Milby via use-livecode <use-livecode at lists.runrev.com> wrote:
> 
> Is there anything posted that gives an overview of how the source code to
> LiveCode works/is organized?  I know that the source is all there and I can
> read it, but that would take a long time :)  I'm still learning C++ (been
> many years, but I learned C while in college), but following the code
> doesn't seem too bad.
> 
> I'm specifically looking at the parsing of livecodescript stacks with
> BOMs.  I think I know where the file is ultimately processed (the stream of
> it anyway), but I can only work backward so far and get stuck.
> 
> So, I guess a more specific question would be where to start on the handoff
> between the IDE and the engine.  As an example, I can see that when you
> turn off debug mode, the message box ends up with "send script pScript to
> pObject".  I want to follow the problem statements through to see where
> they lead and compare what works to what doesn't.
> 
> I'm using the message box (single line) to test syntax.  I get the same
> results whether debug mode is on or off.  The local file includes the UTF8
> BOM (verified by getting the first 3 bytes).  The actual file I'm using is
> the Scriptifier (URL from an earlier thread).  I get the same results for
> "binfile:/" and "https://" forms, so I've only included one below.  I
> observe that "url" keyword is required (will need to adjust the
> documentation syntax, looks to be the same back to at least 7).
> 
> So far I've observed the following fails silently (no error message
> returned):
>  go [to] url "binfile:/...livecodescript" --should work
> 
> The following fail with an error (LC9 gives "message" as hint, LC8 gives
> the actual script line as a hint "go..."):
>  go [to] [stack] "https:...livecode" --missing url
>  go [to] [stack] "binfile:/...livecodescript" --missing url
>  go [to] stack url "binfile:/...livecodescript" --should work
> 
> These work:
>  go [to] [stack] url "https:...livecode"
>  go [to] [stack] "/(full path)/Scriptifier.livecodescript"
>  go [to] [stack] byte 4 to -1 of url "binfile:/...livecodescript"
> --assumes UTF8 BOM
> 
> The current develop build has the same issues (at least for me).  I can see
> in the code where it looks like the BOM should be accounted for
> (MCDispatch::doreadfile or MCDispatch::trytoreadscriptonlystackofsize
> depending on branch).
> 
> It looks like the URL keyword will correctly parse a binary stack, but
> fails on livecodescript with a BOM.  I put 2 files on a server that I
> control (with and without the UTF8 BOM), and the livecodescript will load
> fine from https:// if no BOM is included.
> 
> Apologize for the long post, but wanting to contribute to the community if
> I can.
> 
> Thanks,
> Brian
> _______________________________________________
> use-livecode mailing list
> use-livecode at lists.runrev.com
> Please visit this url to subscribe, unsubscribe and manage your subscription preferences:
> http://lists.runrev.com/mailman/listinfo/use-livecode