Increasingly, we rely on software to automate our lives. “Hey Google,” we say, “what’s the weather going to be like today?”…“Alexa”, we call, “order me a new dishwasher”. In the workplace, we use automated HR services to check through applications at breakneck speed, create chatbots to answer all of our customer’s burning questions and rely on automation tools like Zapier to boost our productivity.
Every task we do seems to be heading towards automation. The robots are taking over! But is QA automation also the way forward? Can you really automate everything?
What is automated QA testing?
Automated QA testing uses specialist software to execute pre-scripted test cases. This means that you can run thousands of tests efficiently by programming an automation tool. For example, your new social media app may require a registration form with multiple-choice questions. A script could automate each answer to make sure that all options perform correctly. If the outcomes don’t match the script, they would be flagged for review. In cases like these, automation can be a huge time saver versus a human tester.
But can you automate everything?
Automation is an important part of any testing toolkit. If you know exactly what you want to test, you can effectively achieve a pass or fail scenario, and quickly identify any problems that may arise. But can every part of testing be automated?
Dan Ashby, former Head of Testing at eBay, answered this perfectly when he spoke to Global App Testing co-founder, Ronald Cummings-John for our Amazon best-selling book, Leading Quality. Dan argues that there are two types of testing:
- Investigating: using our ideas and creativity to attempt to uncover information about a product.
- Verifying: confirming or denying our pre-conceived expectations of how a product will behave.
For example, if you wanted to conduct investigative testing on your new hotel booking app, you might manually test by attempting to book a room in Paris for the following month. In turn, you may discover a bug you hadn’t expected. This will then lead to verifying testing to confirm whether this is a common issue when trying to book a room. If similar issues arise, this will provide more information to guide the investigation.
“The problem with verifying activities is that you can only verify what you know needs to be checked. But once a problem is uncovered, you then have to investigate it. And that is why you will always need some form of manual testing.”
When software engineers create apps, they can write up a checklist of all the things it should be able to do. A QA automation specialist can then turn that checklist into a test to verify that the software can do everything it’s supposed to. But, a level of creativity is required to identify ways in which software may not work, and to think up scenarios whereby programmes may not perform as expected.
There will always be unexpected and unknown possibilities in software development that need to be explored. If you can’t plan for it, you can’t create an automation script for it. Therefore, you will always need professional testers who can use human imagination, knowledge, and experience to conduct investigative testing. Without them, you cannot prepare for the unexpected.
When should you automate?
It isn’t currently possible to automate everything in QA testing – so when should you automate?
Let’s look at the research. In a study conducted by IBM where researchers calculated the cost of manual testing vs. automation, it was found that there were three main situations where automation was more efficient than manual testing:
- The automated test case is expected to have a relatively long life without needing to be changed or edited.
- The test case is comparatively easy to automate, meaning that it can be created from a generalized manual process; the more complex the task, the more difficult it is to automate.
- The comparative cost of automating is lower than that of executing the test manually.
If these situations are in place, then it will likely be cost-efficient and/or time-efficient to automate your testing.
Automation vs humans?
You wouldn’t ask an engineer “what’s the best way to code?”, because that would suggest they were limiting themselves to just one method each time. So why should you only limit your testing to just automation or manual?
What have we shown?
- There are two types of testing: investigating and verifying.
- For investigative purposes, you need creativity, and therefore require manual testing
- For verifying activities you can use either manual or automated testing to test pass or fail scenarios.
- In certain situations, like those we outlined in ‘when should you automate?’ automated testing can save money and time, and increase productivity
- In other situations, the creativity of humans can help you discover bugs that you might not have expected.
The answer is clear: by using a blend of automated and manual testing, you can find bugs that may impact your customers before they do. Global App Testing combines humans and intelligent automation to deliver fast, accurate, actionable test results – effortlessly.
If you would like to learn more about manual testing vs. automation, read Leading Quality, the software industries’ ultimate guide to quality.
Author: Amelia Whyman
Amelia is a Marketing Executive at Global App Testing, a crowdsourced testing company with 25,000+ testers in 105 countries. She writes about QA, software bugs and quality company cultures. She is interested in the future of autonomous testing.