go back to the blog

‘Dealing with Testing Gremlins Through Agile Testing’ by Anko Tijman

  • 04/04/2011
  • no comments
  • Posted by EuroSTAR

Through Change Management we know of two reasons why people would want to change: if the old way hurts, or if the new way is very pleasurable. Often the change is driven by a combination of the two: leaving the old and good expectations for the new. In the testing sphere we have a great challenge looming: that of realizing the wish of many organizations to implement Agile. In this column I examine how agile testing deals with traditional ‘gremlins’ and which benefits the new testing approach brings.

Dealing with Gremlins

Of course, I started in traditional test projects. Although… my first project in 1997 was millennium testing at a bank, and there was a working system in production but no corresponding test documentation. Sound familiar ….? But even in well set up V-model projects we ran into the limitations of the traditional testing methods. Those methods prescribed that testing had to be independent, that it took place after design and development and that an independent test manager was responsible for the quality of the product and for guaranteeing the test process. In practice there were several problems: there was a lot of miscommunication between design, development and test, testing was on the critical path and sometime the client was shocked by the product that was delivered. Conversations SIG testing nights soon lead to the conclusion that these “gremlins” were generic: many test projects were affected to a greater or lesser degree.

Building Bridges

You quickly realize that communication is central to everything: to your team, the project leader and the client. So you build bridges with designers and developers. The designers determine your specifications and, as a clear statement from test guru Boris Beizer says: “The design isn’t done until the test cases are ready”. By providing input into the specifications, from a testing standpoint, the system was better thought out and also easier to test. To be honest, I have seldom been able to do that successfully by invoking the test strategy and claiming that the testers had the right to review their documents. What I did do was, in a good conversation, emphasize the benefits for both parties and bring up the risks associated with not reviewing. In a dialog you achieve far more if you approach it from a common interest.

Extra deliveries

Developers have code that you have to test and they base themselves on the same specifications. So if you cooperate,  on the one hand you can learn more about how they interpret the specs and on the other hand give feedback more rapidly about how their code is performing. As a tester you give, in fact, insight into the quality and actual progression of the project. So as a test coordinator, I have frequently asked for extra code deliveries, even when those extra deliveries were not part of the project planning. This allowed tests to be carried out early, and we had more control over the delivered quality. We increased the number of feedback loops.

Client interaction

If the client discovers the product that you deliver isn’t the product that he needs, then it appears that the time in which you build and test and you had no interaction with the client, was actually a lost opportunity for improvement. It seems that you could do much better creating interaction moments during those phases. It is important that you manage expectations during that period and above all: keep the communication channels open. Even without allowing functional changes, you can drastically improve the acceptance rates during a traditional project. By having the client take a look, with more realistic expectations, during the system test for example. They gain understanding, you get feedback. Laying down broad communication channels with the client is essential if you want to be successful as a team.

When you realize that managing these risks, these traditional test gremlins, is central to a new approach called Agile software development, you will be, just as I am, convinced that this approach can lead to more successful projects!

Enjoying new things

Besides solving old problems, agile testing offers new benefits for testers. The most important difference that I experience in the practice of agile projects is that a multidisciplinary team takes on a commitment to deliver a quality product. It is all about the Definition of done – the exit criteria of the iteration. It summarizes what the team itself thinks that it should deliver, in terms of working software and of course what the client expects. These criteria can range from the unit test coverage to an installation guide. With that commitment comes a shared responsibility to deliver a proper product. From this commitment, it is much easier for every team member to emphasize on doing the best job you can, as a team.

From function to role

As a tester this means that your job function will increasingly become a role: everyone in the team will do some kind of testing as part of his or her work. This also means that your role as tester will evolve towards knowledge sharing (test techniques for instance) being the coordinator (test strategy, test phases) and communicator (with the customer on design and acceptance). This shared responsibility for testing and quality is something that has seldom come forward in a traditional testing project. This was caused by the fact that phases and activities were divided in time. But now testing finally plays the important role that we as testers were always hoping for: being an integral part of software development, adding value at every step of the process.

Paradigm shift

For many testers, this new role is quite a paradigm shift: instead of taking responsibility for the testing, now it’s your job to share that with non-testers. This means that you have to focus more on the other team members, from other disciplines. You’re going to have to interact with your team members about the testing process and how the act of testing can provide value for them. I’ve talked with many non-testers about testing and for them testing actually is only one simple thing: feedback. It’s simply knowing what test criteria should be met, before you actually start and checking whether that is the case. This counts for requirements as well as for code. Writing test cases that go with the requirements up front is called Acceptance Test Driven Development. And writing unit test cases up front at code level is called Test Driven Development. This is actually nothing really new, but in agile projects these effective measures are much more achievable through the joint commitment on quality. Testing is knowing it for sure!


When you start helping your team mates with testing, then your communication skills need to be okay. But most of the success comes from your own mindset! Do you believe in the power of multidisciplinary work? Think of a football team, where defenders and attackers have the same goal! Are you willing to challenge your own testing perspective by talking to designers, developers and clients? Can you clarify in a careful way what they are missing when they’re not testing? Above all, remember that you all have the same goal, just like a soccer team.

I hope that I have lifted the veil around the possibilities of agile testing. It helps you tackle traditional testing gremlins and offers opportunities for significant improvements when it comes to testing being an integral and valued part of system development. Want to try, or not yet? Please leave your comments on our blog!
Anko Tijman is Principal consultant Agile testing at Ordina. You can follow him ontwitter (@agiletesternl) and he holds a (mainly Dutch) blog:

If you have any comments on Anko’s article please let us know in the comment box below. We would love to hear from you!
agile content CTA

Blog post by

go back to the blog


Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery