Open Letter to Rev: Quality Is Job #1
wjm at wjm.org
Fri Oct 20 03:38:48 CDT 2006
Why is Rev's quality control going to hell?
In my opinion, the 2.7.x series has been a huge step backwards in terms of
reliability/stability and usability. And after five revisions it's still not
ready for prime time. Here are some glaring, show-stopper problems that
still persist in 2.7.4:
- The installer introduced in 2.7 is horrible. It installs desktop shortcuts
that point nowhere. The file associations are busted. It doesn't perform
uninstall correctly. The full program directory is present after uninstall,
junk is still sprinkled through the registry, and Rev doesn't remove its
entries in Add/Remove programs. While the 2.6.1 installer works perfectly
under Vista, the 2.7.4 installer locks up and has to be forced-quit. Error
handling within the installer is a joke. Install/Uninstall should be one
thing that is bulletproof, but these apparently haven't even been "smoke
tested." Please tell me ... Aside from copying bits to the Program Files
folder, what does it do RIGHT?
- Standalones created with 2.7.x are bigger, slower, consume more resources,
and are highly unpredictable compared to standalones created with 2.6.1.
Half the time, quitting the standalone leaves the process running in Task
Manager -- when the exact same stack under 2.6.1 exits cleanly. We've had
reports of libraries not being included even when explicitly added. And Bill
V's recent post is not the first report of standalones being built
incorrectly or bugs returning from the dead.
- More often than I'd like, quitting Rev 2.7.x itself results in an
application error upon exit. And that is if you're lucky and the process
isn't simply "hidden" as it is in standalones that don't quit properly. When
the Windows Error Reporting dialog comes up I click "Send Report" every
time... do you guys even receive/read those reports? Because I haven't seen
any decrease in their frequency from one release to the next.
- The IDE is noticeably, painfully slow, and suffers from screen refresh
problems and on-screen corruption. I've had the 2.7.x IDE (and my work
product) simply disappear more than once. The sluggishness of the current
script editor is simply unacceptable (as described previously). It makes me
long for the days of editing on a ZX81 in SLOW mode. This is not "software
at the speed of thought" -- it's software at the speed of "musing."
- Geometry manager still is hit-or-miss. It works until it doesn't... then
you have to rebuild everything from scratch. One prominent developer told me
he never uses it, because of the peril to his work product. Just when are
you going to get around to fixing that?
- The XML library under 2.7.x seems to consume much more memory and not
clean up after itself. Stacks that use XML work fine under 2.6.1, but have
locked up with 2.7.x.
- The Internet library has resulted in lockups in stacks that have run
without incident under 2.6.1.
In fact, the 2.7.x series is so awfully bug-ridden, its most welcome new
feature is the ability to save in "legacy" format.
Yes, Rev, it's really that bad.
I wrote a long time ago that I planned to purchase every update to the
product. That's because I genuinely want to see Rev succeed (not to mention
continue to exist/pay the bills). But I have to take it back. No, not this
time. Not with 2.7.
See, your "subscription" approach to licensing scares me. I just don't think
you'll be able to release a sufficiently bug-free version within the 12
months of the update pack duration. It's already been eight months since you
released the first version of 2.7, and admit it -- it's really still not
up-to-snuff. If I had purchased the pack in February, when my 2.6.1 updates
expired. I'd have regretted it. In a traditional update/upgrade model I
might have the expectation that you'll eventually get 2.7 "right" via free
bug-fixes and you'd already have my $150/$199. But under this model I'm not
giving you a single shilling until I see a release that installs correctly,
builds correctly, keeps up with my typing, and doesn't crash half the time I
Bottom line is you haven't EARNED the update pack fee yet.
Some of you will say, "Bill it would be much more productive if you filed
all this in Bugzilla." Well, I'm all for contributing to the community. But
it takes time and effort to file a decent bug report. You need to have
something reproducible, supply sample files, write it up properly, etc. This
is no trivial task. And I just don't feel that it's worthwhile. I haven't
seen action on other serious bugs, and I haven't seen the kind of quality
that suggests even casual inspection on the part of Rev's release team. If
the steering wheel comes off in your hands, are you going to take the time
to write about the radio not working? How could you NOT notice that your
installer puts a generic icon (broken shortcut) on the Windows desktop? How
could you release a script editor that is slower than molasses? Where is the
pride in your work product?
Besides, I don't even "own" this version. After every announcement I
download the latest draft; I test it out by converting some working 2.6.1
stacks; I see the problems almost immediately; I "uninstall" (that is, I run
the uninstaller, see it's still screwed up, and manually go through the
files and the registry to clean up your mess); and go back to 2.6.1. I'm
afraid 30 days of trial isn't enough time for me to thoroughly document all
the things that are screwed up in this product. So, when YOU demonstrate
some good faith -- releasing a reasonably robust product, and committing to
free bug fixes -- then perhaps I will work as your unpaid Quality Assurance
Hoping 2.7.5 is the one that gets it right,
More information about the use-livecode