go back to the blog

G(r)ood testing 15 : The three + one lessons of test automation

  • 12/06/2015
  • no comments
  • Posted by Derk-Jan de Grood

In June 2015 I spoke at Test Automation Day (TAD). The aim of this conference is to promote test automation and to exchange experiences on how to implement it. I have spoken three years in a row at this conference. What I really like about this is that my story grows every year. The development of my story keeps pace with my own learning curve. And the stories seem to become a sequel. In this column I give a brief summary of the overall storyline.


Lesson 1: It’s not an option

Eight years ago test automation was something for a small group of technical testers. The bulk of testers preferred manual testing. I noticed a lot of cold feet when I suggested to start with test automation. The general believe was that Automation would only flourish in mature organizations. So many testers could conclude after a brief maturity assessment that the organization was just not yet ready for it. This way they could keep the technology out the door and continue to do what they were good at: manual testing and start test processes improvements in order to fill in the missing preconditions.


Nowadays we changed our way of thinking. Test automation is not an option. It’s a necessity. Better tooling and cooperation with developers lowered the threshold to get started. On my first TAD I shared a customer case where I was asked for an advice and told them that they were not ready yet. In this particular case I still don’t believe that automation would have been successful, but I rather had said the opposite: “Go ahead and start learning.” It takes a while to implement Test Automation on a large scale, and people need to learn and find their way. The best way is by doing it, by experimenting and tuning the processes so it works in the context of your organization. Lesson one is therefore, don’t hold back, when the organization has not fulfilled al prerequisites. Get started !


Lesson 2: It won’t fly without a business case

After this lesson, I was ready for my next lesson. The IT engineers were convinced and wanted to automate their tests. But we needed commitment and budget from the management. Managers reason differently and rather than technical arguments, they think in business case. So we need to explain how the investment contributes to the business objectives. Of course we could state that test automation leads to faster and better software. But for C-level management that won’t be convincing enough. We need to quantify our argument, estimate the required investments and predict the benefits. It requires some creativity and math to create a convincing business case. My experience is that especially technical engineers find it difficult to leave the world of technical certainties and enter a world that is based upon estimations and best guesses.  We should be aware of this and offer them some help. In 2013 I therefore presented a checklist with precondition for test automation. It showed the relationship between the maturity of the organization and the required investments that can be included in a convincing business case.


Lesson 3: TA is a driver for change

Last year I took the story a step further. Together with Ard Kramer I shared our experiences with the actual implementation of test automation throughout the organization. Preconditions still need to be met and business cases are still being made in order to get management support. But our next challenges were amongst others; tool regulation, knowledge sharing and integration

It fits the agile mindset to let each team determine what tool is best for them, but what are the costs

associated with a sprawl within the tool landscape.  How can we secure central knowledge and avoid that a team wastes a lot of time and money reinventing the wheel. In a large organization it is likely that other teams already have a lot of experience with automating the same kind of interface.


We decided to get started and learned that test automation serves as a trigger for improvements. Because we were starting with test automation, the teams were making huge growth spurts. Suddenly, there was a need to capture the regression tests. Before they were done without any documentation and depended very much on single professionals. Automation also triggered the question which tests to automate first. This forced the testers to make a shift left and engage with the business.  Operational awareness was created when we asked the project members who would own the regression test-set after the project was done. Ops will run the automated tests during the entire lifespan off the system. It’s only logical that they have requirements concerning the integration of new tests into the existing test set. When the project delivers their tests in e.g. a new tool that does not integrate, extra operational cost will be the result. Defining these roles clearly and discussing expectations over the boundaries of the individual projects clearly was a turnaround. Roles and responsibilities were exchanged and the organization got a far better view on the objectives they were pursuing together: Quality solutions with a short time-to-market and low operational costs.


Lesson 4: The full circle takes you back to earlier stages

This year we learned yet another lesson. The story is not over yet. What do you do, when you got the organization aligned; when you got buy in from the management, but the testers don’t know what to test. What do you do when you got your tools in place, but teams don’t know how to test? Maybe it is about maturity after all, and we are back to square one. This is what we call the full circle. It will be the topic of this year’s contribution. I am looking forward to sharing our forth lesson.


Blog post by

go back to the blog

Derk-Jan de Grood

Derk-Jan de Grood works for Valori as senior test manager and Agile Transition Coach. His drive is to improve the visibility of testing in both agile and traditional organizations, by sharing his knowledge and experience by means of training, presentations, workshops and publications. He is a regular speaker at conferences like EuroSTAR, wrote several successful books on software testing and publishes articles for the major testing magazines. In 2014 he won the EuroSTAR Testing Excellence Award.

Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery