Blog

go back to the blog

Cost of Automating Manual Tests

  • 16/10/2013
  • 6769 Views
  • no comments
  • Posted by EuroSTAR
-->

Great focus is put on automation of manual tests these days and more and more people are asking the question – how much will it cost? We’re not talking about the price of a tool (although some are super expensive and some are free), we’re talking about the total cost. Regrettably the answer to this question is still – it depends. However, no matter if you are outsourcing or doing it yourself, the biggest cost of automating tests is still time. Planning and creating the suite but maybe even more important, maintaining it?

Test automation of acceptance tests currently has bad reputation and most of the interviews we have made tell us that it’s because of the effort and cost in maintenance. Almost all the people we have spoken to that have tried acceptance test automation state that they have experienced failed projects along with huge costs in the past. Some argue that it’s the tools fault, that it’s too hard or costly to create and maintain the test scripts or that it’s impossible to tell whether it’s the tool or the application that’s source of the problem. Some say it’s because of bad planning or lack of a good processes. We think it’s all of the above and not just any one thing…

We have seen that it takes 2-3 days of workshops or training for experienced testers to be able to use our tool – that’s good progress BUT it doesn’t consider the fact that, in order to make a successful automated test suite – you need to really plan and think about what to test and why. It might be easy to use a tool but it’s generally not easy to do great test automation depending on what you need to achieve.

We collaborate with Chalmers Technical University to advance the state-of-art in automated System and Acceptance testing. The initial research shows that using the graphical user interface based test technique Visual GUI Testing (VGT) can greatly improve test frequency of System and Acceptance test regression testing and thereby lower cost and raise software quality. In addition, it is perceived that VGT doesn’t demand the same level of programming skills as some of the tools using older technologies (QTP, Selenium ect.)

Our experience shows that you can probably automate most manual test cases with VGT but the question is – is it cost efficient to automate everything? Isn’t automation supposed to give more value than it takes? What can be done to make it easier, and in the end lower the cost of acceptance test automation?

Conclusions:

1. Decide why you want to test (which problems will you have if you don’t do tests and how will that affect your business?)
2. Decide what you need to test in order to fulfil 1.
3. Decide how to test it in order to fulfil 1.
4. Choose a tool that suits your needs.
5. Evaluate the cost and the value- is it worth it?

…and be sure to plan for maintenance of your suite before you start creating scripts.

Author: Olle Alvemark

olle alvemark_100x147Currently living in Gothenburg but grew up in the north of Sweden. Master’s degree in automation at Chalmers University of technology. Gave up a career in energy performance contracting to pursue making Visual GUI testing and JAutomate a part of the global testing community. Feels for the human aspect of test automation and is keen on making tools fit the human and not vice versa.

Read more about their tool JAutomate at http://www.jautomate.com/

Or visit us at Stand 31 in the EuroSTAR Conference Exhibition.

Blog post by

go back to the blog

eurostar

Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery