Revolution 2.8.1: a 240+ bug fixes/improvements! What about the remaining 1879?

[Bill Marriott]

I'm not going to argue that there are still too many unresolved bugs. There 
are.

However, there are several facts that seem to be overlooked, ignored, 
discounted, or chosen selectively by some:

1) RQCC is not authoritative.

- There are many open reports that are fixed but not actually marked as 
such.
- There are many reports miscategorized. Such as those marked "critical" 
just because someone thought this would get more attention or was critical 
*for them* but are not actually critical in the sense of data loss or 
crashing.
- Not all bugs are created equal. Some affect most users; some affect only a 
small number of users. Choosing what to work on first is part of the quality 
improvement process.
- A substantial number bugs are discovered and fixed within the development 
process without ever being reported in RQCC.
- Human/time limitations have prevented 100% comprehensive logging of 
activity in the system. Engineers have looked at issues that are still 
marked "unconfirmed" for example. There is steady effort to improve on this 
aspect.

2) No one ever said we're done.

- The version is "2.8.1" not "2.9"
- Linux support has been going on quietly behind the scenes throughout the 
open beta period. Many bugs are caused by code which is specifically being 
targetted for Linux, and those will resolve with the new engine. It makes no 
sense to fix/test those changes twice.
- Many other "generic" bug fixes -- ones that do not depend on Linux 
specifically -- are still in process and will be seen over the next series 
of betas.
- Even when version 2.9 is released, it will still have bugs. Every product 
more complex than a coffee mug has bugs.
- Some bugs are next to inscrutable. All are submitted by users who do not 
have formal training in testing or bug reporting. Every report is a 
considerable effort to understand, reproduce, and research, even if it's 
written well. It's a significant effort to work through RQCC items no matter 
how you slice it.
- The pace of bug fixing may not please everyone, but it is proceeding 
slowly and steadily.

3) Revolution 2.8.1 is a marked improvement over previous versions.

- 240+, 430+, 753+ -- There have been a lot of numbers thrown around. 
Whatever number you choose for resolved/fixed bugs, it numbers in the 
hundreds. RunRev marketing materials use the most conservative number, by 
design.
- The effectiveness of the process is best demonstrated by the perceived 
quality as reported by users. This has been overwhelmingly positive.
- The surveys back it up in a more empirical way, with EVERY measurement 
showing a 15% to 20% improvement over the baseline Beta 1 survey, and in 
many cases far exceeding that.
- More users than ever before, a solid majority, rate 2.8.1 as preferred 
over 2.6.1 (which I think is fair to say was the previous standard of 
stability).
- The integration of the new externals, especially the Browser and Database 
facilities, addresses many enhancement requests and provides solutions to 
previously impossible needs. They are definitely about Quality.

4) The "Open Beta" process is working well and going in the right direction.

- We have more users than ever enrolled in the RQCC.
- Those users are more active than ever before, filing more reports and 
comments.
- We have more beta testers than we ever have. Nearly 550, with more 
applications coming in regularly.
- Those testers have had a longer test period -- nearly six months -- to 
work with the product than ever before.
- The overwhelming majority of survey respondents rate the beta test emails 
(96%) and thoroghness of communications (87%) as Good or Excellent.
- RunRev made a major investment of time/resources to dramatically improve 
the beta test experience with the release of the new RQCC at the beginning 
of the Open Beta.
- Far from being "secretive" the RQCC is open to everyone, whether they are 
a paying customer or not, enabling the kind of review of warts we're seeing 
in this thread.

5) It is worthwhile to participate, RunRev is listening.

- The RQCC is a friendly, usable system which essentially did not exist 
before Open Beta. (We had Bugzilla but that was virtually unusable.)
- Engineers are more engaged in the bug reporting, research, feedback, and 
fixing process than ever before.
- The "Open Beta" is an effective way to uncover problems in the software 
before major versions are shipped. It did not exist before November.
- The surveys measuring satisfaction have never been conducted before. The 
results from survey #1 were eye-opening. The results from survey #2 are 
vastly improved, but still not where they should be. There will be 
additional betas and additional opportunity for constructive feedback that 
is truly read and considered.
- We've never before asked a large group of users to submit their "Top Five" 
issues directly to a human without going through the bug reporting or voting 
rigmarole... but that's exactly what we did with the Open Beta group.
- It is RunRev itself that has shined the light on quality with these and 
other initiatives, both visible and behind-the-scenes.
- RunRev is still committed to a even-better Revolution 2.9 release, and the 
measures above are the best way to make it happen.