Testing Standards

[Thomas McGrath III]

Mark,

Thank you, this is exactly what I was looking for. I have spent the  
last week and a half going over the specifications and using our Rev  
project as proof of concept. Since we are using an outside company  
for the coding, they have a person doing the use-case scenarios and  
specs. They will also have a person writing the QA tests when all of  
this is done. So far I think we are on track with what you are  
describing here. Having the Rev demo really makes all of this easier  
(for me). I just don't have the working experience in this process  
and am a little nervous. Things like what are the next steps and are  
we on track and is this enough to describe etc. are what is  
concerning me.

Do you recommend any 'good' books on this process? Maybe on what to  
expect in the specs? Maybe what is most often left out and or what  
most often causes the most problems?

Thanks again Mark,

Tom

On Nov 9, 2005, at 2:16 AM, Mark Wieder wrote:

> What's worked best for me is a hard one to get everyone to sign onto,
> but it's a matter of *not* coding and *not* writing tests until the
> last phase of the project, but rather putting most of the effort into
> getting the engineering specification document defined. Once that's
> rock-solid and all the ambiguities are out of it then engineering can
> start building to match the spec and QA can start building tests that
> also meet the spec. Once engineering starts delivering builds to QA
> then the tests should run without failure. When bugs are discovered
> they are either the result of a) engineering not building in
> conformance with the spec, b) QA not building their tests properly, or
> c) an ambiguity in the spec that wasn't caught in the design process.
>
> The a) and b) bugs are easy to sort out and assign to the proper party
> for resolution. The exceptions are what are then brought to the bug
> meetings to be resolved.
>
> When the inevitable changes occur over the design life-cycle, those
> changes are incorporated into the specification and both engineering
> and QA make the appropriate modifications.
>
> This design philosophy is at the opposite end of the spectrum from the
> Agile Programming folks. I spent a year and a half as QA lead on a
> project following this method and delivered two award-winning products
> reasonably on time (changes in requirements mid-project - gotta love
> those marketing types) and within budget. The reason it's hard to sign
> onto is that it requires a leap of faith in the process - people are
> itchy to code and there's no product to show for maybe the first two
> thirds of the develoopment cycle.
>
> With rev there's a bit of a difference, since the product can be
> storyboarded as a proof of concept for the client and then the
> prototype later becomes the working model without having to start from
> scratch again.
>
> -- 
> -Mark Wieder
>