hair-pulling frustration

[Richard Gaskin]

Éric Miclo wrote:

 > I do agree that more testing (non automatic testing) should be made
 > before releasing a commercial product.
 > But I’m not sure RunRev will listen.
 >
 > When the CEO says:
 >  "I really would urge everyone in our community to move to 7 as
 > soon as possible. We spent 18 months building it and considering
 > the amount of change its getting very good results stability wise.
 > There are issues but they are mostly minor."
 >
 > When minor bugs are so many, I don’t consider it is minor to test
 > and fix before releasing a commercial product.
 > And by fixing I mean test if the fix really fixes the bug so it
 > doesn’t have to be reopened immediately.
 > It is not the amount of time spent on a product that is an indicator
 > of quality.
 >
 > My 2 cents.
 >
 > Best,
 >
 > ÉrIC
 >
 > -- My NeXT computer will Be a Mac too! --


Re/Code recently posted the full video of their interview with Apple VP 
Greg Joswiak, discussing iOS 8.0.1 bugs among other things:
<http://www.macrumors.com/2014/11/12/greg-joswiak-full-interview/>

iOS 8.0.1 was sent out in response to critical bugs in v8.0, and it 
turned out that the fix itself required fixing, as 8.0.1 wound up 
bricking some user's phones.

In the video the interviewer raises the larger question of overall QA, 
and we're all familiar with recent issues there from iMaps forward.

Watching the video, I don't feel Joswiak's obvious pride in Apple is at 
all misplaced.  Sure, there have been many issues, sometimes critical 
ones, in both iOS and OS X in recent years, and this may seem surprising 
given the wealth of resources Apple has at their disposal which would 
seem more than sufficient to prevent such things.

Software is complex stuff.

I don't think it was a mistake for Tim Cook to suggest we upgrade our 
Apple devices to the latest software version which later bricked some 
phones, and I don't think it was a mistake for Kevin to suggest that we 
use the latest version of his company's software.

Obviously, with the benefit of hindsight, both CEOs were mistaken about 
the fitness of the final deliverable, and both companies have devoted 
considerable resources to addressing the issues once they became evident.

I'd wager that Apple has now added additional QA review steps to prevent 
the specific issues that affected iOS 8.0.1, and I'd wager RunRev has 
done the same with the issues evident in 7.0.

Late-stage regressions are especially troubling for all of us. 
developers and users alike, because in the late stages it's especially 
hard to detect such things as might have happened during earlier Beta 
periods.  When unpredictable events occurs late-stage it becomes more 
likely that a software will be delivered that makes standalones unable 
to quit, or bricks people's phones.

I wouldn't assume that either Apple or RunRev spends nothing on internal 
testing.  On the contrary, as a percentage of payroll I'd be surprised 
if RunRev's costs for that weren't a multiple of Apple's.

I volunteered for this role of Community Manager because LiveCode is 
very important to my own business interests and those of my clients. 
Part of this role is to advocate for community concerns, and I agree 
that QA is critical.

We *all* want to see QA improved, ever more so going forward.  The folks 
at RunRev have more at stake in this than most of us.

Those of us who read the bug DB regularly have a good appreciation for 
the level of engagement the core team has in the QA process.

That v7.0 shipped with show-stoppers is annoying to all of us, and 
exploring ways to improve the QA process is the focus of my meeting with 
Ben later today.

Sean's suggestion of including a reporting item in the Help menu is a 
very good one, and I'll be sure to raise it at the meeting.

Any other specific actionable items like that are very welcome, and can 
help us all refine the process of making complex software ever more robust.

-- 
  Richard Gaskin
  LiveCode Community Manager
  richard at livecode.org