Testing Standards

[Mark Wieder]

Tom-

Sunday, November 6, 2005, 8:02:12 PM, you wrote:

> Thanks for the kind words and comments. I am now trying to set up a
> testing team for my project. And, well, I am used to doing this as a
> one man team and i am pretty good at that, but I need to organize a
> small team of 2 to 4 people to do the testing on the ANSI C  
> engineering aspect of the project and am not sure of the best way to
> go about this.

> Is there a set of standards for group testing? What about software? I
> know our programming team is using bugzilla, which is great for bugs
> but what about features and whether they are being done right or not?

If you're using Bugzilla and happy with it, then stay there. Feature
requests can be accomodated, as can feedback on features and the lack
thereof.

> Does anyone have some pointers in this direction? How to keep the  
> team together and on the same page?

Well, nobody else has chimed in on this so here goes...

What's worked best for me is a hard one to get everyone to sign onto,
but it's a matter of *not* coding and *not* writing tests until the
last phase of the project, but rather putting most of the effort into
getting the engineering specification document defined. Once that's
rock-solid and all the ambiguities are out of it then engineering can
start building to match the spec and QA can start building tests that
also meet the spec. Once engineering starts delivering builds to QA
then the tests should run without failure. When bugs are discovered
they are either the result of a) engineering not building in
conformance with the spec, b) QA not building their tests properly, or
c) an ambiguity in the spec that wasn't caught in the design process.

The a) and b) bugs are easy to sort out and assign to the proper party
for resolution. The exceptions are what are then brought to the bug
meetings to be resolved.

When the inevitable changes occur over the design life-cycle, those
changes are incorporated into the specification and both engineering
and QA make the appropriate modifications.

This design philosophy is at the opposite end of the spectrum from the
Agile Programming folks. I spent a year and a half as QA lead on a
project following this method and delivered two award-winning products
reasonably on time (changes in requirements mid-project - gotta love
those marketing types) and within budget. The reason it's hard to sign
onto is that it requires a leap of faith in the process - people are
itchy to code and there's no product to show for maybe the first two
thirds of the develoopment cycle.

With rev there's a bit of a difference, since the product can be
storyboarded as a proof of concept for the client and then the
prototype later becomes the working model without having to start from
scratch again.

-- 
-Mark Wieder
 mwieder at ahsoftware.net