On the Democratic Operation of Bugzilla

[Alex Tweedly]

Garrett Hylltun wrote:

>
> On Feb 23, 2006, at 11:07 AM, Dan Shafer wrote:
>
>> This provided me with an opportunity to say something I've been
>> meaning to say for some time but never had a "trigger" for.
>
>
> Ditto!
>
Yeah - I tried to respond to Sarah's email saying more or less the same 
thing, but just couldn't express myself well enough, so never sent it. I 
think the bug-voting system should never be more than a tiny hint to the 
people setting priorities within RunRev.

> <snip>
> But what upsets me the most is depending on the paying customers to  
> help them track down bugs!  What the hell are they doing with the  
> money?  And what the hell are they doing releasing a product that is  
> already known to have bugs still in it!   They should be paying  
> testers for this and not raping the paying customers for this work.   
> With the prices they are charging for everything, we shouldn't even  
> be having this conversation at all!  If a user finds a bug, he/she  
> should be able to simply report the bug to Rev either via email or a  
> bug report form on their site, and they should take care of  
> everything from there!  That bug should be gone by the next update of  
> the product.
>
I think it's impractical to say that all known bugs will be fixed. Not 
all can be reproduced reliably, bugs reported late in the day can't be 
fixed without delaying the release, some bugs are unimportant and can 
reasonably be left until other work is to be done in that area of the 
code, etc.

And more importantly, there is plenty of evidence that people buying 
software will prefer software with new features and some bugs over 
software that is bug-free but lacking features. So too strict a "fix all 
bugs" policy is a sure way to fail to attract customers (as is too 
strong a tendency to add features without fixing bugs). Small products 
can hope to achieve this, large ones can't. RR needs to strike a 
balance; and while I think they are not getting it quite right, I can't 
say that they're getting it all wrong either.

> I'm sorry for being a bit over the edge, but I've been in this  
> business myself, and this really makes me mad.  You don't release  
> products if you know it still contains bugs!  You don't upgrade your  
> product unless the upgrade fixes all the prior bugs.  Updates are to  
> fix bugs and issues that you didn't catch earlier, that somehow got  
> past your beta testing team, and updates are free since you're fixing  
> your own mistakes, not mistakes of the customer.  Upgrades are not  
> like going from 1.1 to 1.2, but from 1.x to 2.x  Upgrades are when  
> the product has had some major changes done to it, improvements and  
> new features over the previous version.
>
> I'm really starting to regret my purchasing Rev now.  I'm feeling  
> like I've been ripped off.  Rev is a nice product, but if this is how  
> the company is going to operate, then I'm not going to be updating/ 
> upgrading.  And I doubt that Rev is going to change their business  
> practice since it seems so many people tolerate it and  continue to  
> give them money for releasing a product that will always have bugs in  
> it.
>
> Runtime...  Stop charging for updates!  Fix all the bugs and release  
> an update, then work on an upgrade when all the bugs are fixed.  Go  
> ahead and charge for upgrades.

I think each release I've seen (only been 18 months) has been a mix of 
bug-fixing and new features, which is the way I think it should be. I'd 
like to see more clarity on this (e.g. a list of BZ numbers fixed in the 
release). But I do believe that each release has had enough features to 
justify an upgrade fee. While I do find the on-going cost a bit high, 
that's a business decision that RR needs to make (and which I knew about 
when I got involved).

>            Dump the 'zilla stuff and setup your   own internal bug 
> tracking system so you guys can take care of this  and leave the 
> customers out of the process.  Beat the crap out of  your beta testing 
> team for allowing all this stuff to get through to  the customers.
>
I strongly disagree with some of this. I think that keeping the reported 
bug list public is very much the right thing to do. While it can be an 
aim to find bugs internally or in Beta testing, it will never completely 
succeed; customers will find bugs, and it's important that there is a 
process that allows them to report those, track the progress against 
their reports, and see when the problems are fixed.

If some other customer (or even internal user) has found a bug, then I 
want to get the benefit of that knowledge. If there is a workaround, or 
even just a more precise description of the problem, then it can be very 
useful to me. Some bug entries in BZ include discussion or clarification 
from the RR team which is very useful.

I think RR could do with doing a serious review of the outstanding bug 
list. I suspect that 20% of the open bugs could be simply dismissed (as 
"cannot reproduce" and unless there is further input are not going to be 
looked at again), and a further 20% could be amalgamated (duplicates - 
but valuable info in more than one report). (Obviously those are wild 
guess numbers - but I've had lot of experience of dealing with bug 
reporting systems).

Also, I'd like to see a better separation of bug reports from 
enhancements requests. Make a wild guess at 20-30% of enhancement 
requests (i.e. new feature requests, not simply "filling in gaps that 
should have been left in the first place" enhancement requests). Then a 
bug review could reduce the open bug list by 50% - and doing that would 
make BZ into a much more useful tool.

I'd also like for much better summary reporting capabilities to be made 
available to the users of BZ. That would give us a much clearer view of 
bug fixing progress (or lack of it); I hope that would actually go some 
way to improving the feeling people have about bug treatment (though of 
course there's a danger it would instead solidify the concerns people have).

I'd also have a hard and fast rule that NO new bug report should remain 
"untouched" by RR personnel for more than a week. It should be read, 
considered and some minimal remark or state change should be required. 
Failure to do this reinforces the opinion that there is little 
justification for spending time and effort reporting bugs.

And, finally, it's not the beta team that can be faulted - they're 
mostly unpaid volunteers, and had little or no say in whether this stuff 
was released.

-- 
Alex Tweedly       http://www.tweedly.net



-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.375 / Virus Database: 267.15.12/265 - Release Date: 20/02/2006