Testing with a Zone Defense

W14     Start Time : 13:45     End Time : 14:30

Over the years, I have heard a lot of testers suggest writing regression tests for bugs as they are fixed. It’s like a man-to-man defense in soccer (or my beloved American football.) The idea behind the man-to-man defense is that it protects you from suffering the same bug twice. The shortfall is that it doesn’t protect you from new bugs, which are much more likely to occur than previously fixed bugs. My team’s solution to this problem is a “zone defense”, where we break down individual bugs and group them by zone; identify methods to improve our testing of the zones.

In this talk, I will describe the process my team uses to design our zone defense, using examples of defects we’ve seen. We start with Customer-Reported Unique Defects (CRUDs) because the bugs that have made it into a product release often reveal holes in our defense.

Some of the categories of CRUDs we have identified:

  • Simple bugs that we wish we had caught. If you miss these bugs, there is probably an uncomplicated remedy.
  • Bugs that occur with large volumes of data. The first couple of times we saw these issues, we asked, “Who would use our software this way?” Now we know, and we plan for it.
  • Bugs that arise from non-standard workflows. These are “needle in a haystack” issues. We might have found them with deeper exploratory testing, or more finely modularized and randomized automated tests.
  • Bugs that we didn’t know to look for. Some of our bugs revealed a communication problem: functionality was added to the product, but not mentioned in the release plan. We didn’t know it existed.
  • Bugs that no one could have predicted. Sometimes the companies that use our tools are unique and fascinating, and they do things we never anticipated. We have to decide which of these issues to pursue.

Playing a zone defense has taken my team beyond a “good release/ bad release” mentality and created a mindset of responsiveness.

Want to attend? Book your Conference Place

  • Speaker


    Pamela Gillaspie - Test Manager, TestPlant, United States

    Pamela Gillaspie is Managing Test Engineer at TestPlant, maker of the eggPlant range of test-automation tools. At TestPlant, Pamela does functional testing of load-testing software, integration testing of mobile-testing software, and a little bit of everything in between. Above all, she appreciates the opportunity to help shape the tools of her trade. Pamela is a co-organizer of the Boulder QA Meetup Group, and an active member of the Software Quality Association of Denver. She would love to hear from fellow testers on Twitter.