go back to the blog

Reporting the whole instead of the usual “you missed a spot” – by Shmuel Gershon

  • 15/10/2010
  • no comments
  • Posted by EuroSTAR

The role of testing (or one of them) is to reveal information — we don’t create information (though we create ways to uncover it), we don’t make bugs or break software. Bugs and broken functionality are there, we just identify them and catalog them. A Software Quality Information Taxonomy Department.

I wonder howerver why we often narrow our vision to one main kind of information.
We report bugs, defects and issues in a Defect Tracking System. Because that’s what we catalog, that’s what we look for, so that’s what we report. And that’s what we (the whole project/product) acts on.
Looks like we miss a whole lot of informatoin there. One thing we miss, are positive reports.

Software isn’t made only of bad things. There’s a lot of good things too.
Why we report only defects?
Why don’t we see issues reported like “UI-1017: Search menu is a delight to use, great responsiveness“? I think that’s as valuable information as all the rest.

Story: Once upon a time, when fixing a bug, a programmer ended up breaking a different part of the system. I told him “hey, after you fixed the bug on module X, module Y became harder to use”… and the response I got was “you reported the bug, right? I fixed it and that’s what happened, what did you want me to do?”
I bet you had similar conversations too.

But if the programmer had many entries reported to him, one of them said “Mod Y: Good usability on the module, funciton is quick and intiutive” and another one that said “Mod X: Functionality is broken when module Y is enabled“… I believe taking care of not breaking the first report would come naturally when fixing the second one.

Information about good parts of the software is crucial. Having a bug on “Button A: bug so and so...“, wouldn’t it be easy to fix the button functionality in this part of the software if the programmer had as reference that somewhere else in the software there’s a button that “Button B: Good form, function and placement, follow guidelines and app style“?

Maybe we should stop using defect tracking systems and start using quality information report systems.

When we treat quality as “the absence of bugs”, managers start counting them and obsessing with their numbers. But when we have them as part of different information we bring up, managers start looking for patterns in our report. Looks like there can be much benefit there.

James Bach’s Low tech dashboard have a little bit of this, on the smiling parts of the coverage chart. But I’m speaking about more detailed report. And I feel this suits well what Jon Bach advocates in his view that a tester should be like a journalist — giving a holistic report about all aspects of the software, not only one facet. I think it also leverages well Michael Bolton’s statement that software testing role is to shatter self deception (not exact quote… he saidenable everyone to see the detailed implications of their vague daydreams) because it shows people that some parts are good, but they have to pay attention to the rest.

I have to develop this idea a bit more… Can you help? What do you think?

Blog post by

go back to the blog


Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery