Thanks to Eduardo Amaral, Quality Management Director at Noesis, for providing us this blog:
When I hear someone talking about test automation, its benefits, and the investment needed, I always remember an example I’ve recently witnessed: 50% of the regression tests were executed in an automated process for more than a year on a specific eCommerce app from a well-known company. This company had one Automation Engineer working full time on the project to achieve it.
The project also had a quality engineer to execute the remaining 50% of the regression tests needed manually, and all new features. These new features testing represented around 50% of the global execution, which means that the Automation Engineer (a more expensive resource for the project) was only handling 25% of tests delivered. The remaining 75% was executed manually!
It’s a fact that automation is needed and beneficial to the software development lifecycle. It is the only way to keep high-quality assurance standards in a fast-moving market. So, the 1 million dollar question is: What’s missing? Why are most of the companies failing in this automation effort? Why are they just replacing manual testing with automated testing without any efficiency gain?
On the contrary, if we consider that the development costs are higher than manual testing, the reality is that those companies are just increasing costs rather than reducing them (the primary goal of test automation).
There’s no correct answer or a magic solution to make automation happen and work, like in all other foundational changes. Because automation is such a trend and hype in the industry, many companies are just doing it because of it, without having a real need or a structured business case. And that is not a detail. Moving forward with a technical proof-of-concept instead of a proof-of-value is the wrong approach and reveals a lack of strategy!
Achieving success in automation strategy
This lack of suitable strategy regarding the company’s organization and needs, its application landscape, and business processes, makes the adoption of standard automation processes the obvious path – it seems so obvious that failure is not even considered. “If I execute these tests manually today, tomorrow I’ll hire an Automation Engineer and run them in an automated way without additional effort!”
Where can this line of thought fail? It’s simple: we only execute repeatedly (the justification for automation) if we are talking about an application that is constantly being upgraded. Those are the apps that require intense automation maintenance, which also means breakeven slippage and ROI underachievement. And it gets even worse when a manual tester, especially skilled in focussing on business processes and setting up quality assurance strategies according to risk analysis and error patterns, is replaced by an Automation Engineer focused on technology and test patterns.
To achieve success in any Automation strategy, it’s vital to coexist and optimize (not replace) these two profiles – manual tester and automation engineer – during the development lifecycle, taking advantage of what each approach can offer. And most of all, it’s necessary to understand the application’s characteristics and adopt the best automation strategy.
When we’re talking about solutions where time-to-market is crucial, it is impossible to succeed without a seamless continuous testing strategy, where test and validation happen on the continuous integration pipeline. On the other hand, evaluating complex application architectures with significant dependency levels and integration with other systems, such as ERP, Billing, HR, Legacy, etc., makes it difficult to have an efficient use case for automation. The focus must be E2E and only use automation on specific and routine quality assurance processes instead of trying to automate all testing frameworks.
Test automation will be an essential part of any Quality Management strategy in the future and a complement for quality assurance, that’s certain.
But, to have the ambition to automate all the process for every application developed is a dangerous path, with a great probability of not achieving the desired results and producing a financial loss.
Noesis is a Gold Partner at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now.
Author
Eduardo Amaral, Quality Management Director at Noesis
Eduardo has worked as an analyst/developer, project manager, program manager, and service delivery manager. He has a background and certifications in several domains, including quality assurance, project/program management, process management, and product management. Over the last 21+ years, he’s been involved in a variety of software project development lifecycles and solutions integration, at high level of complexity and customer experience exposure. He’s performed in some of the major national and international companies, from different business areas – industry, banking, insurance, telco, retail, etc.