Thanks to Karate Labs for providing us with this blog post.
Test Automation is serious business. With teams under pressure to ship more often for competitive advantage, ensuring the best-quality end-user experience is key.
The choice of a test-automation tool can make the difference between a team that has high velocity – or one that lacks confidence to release.
We have seen many teams that have made the mistake of writing an in-house test-automation framework. I have personally experienced the pain of developers who have inherited tests in such a framework multiple times.
Unfortunately, teams continue to write their own framework for many reasons. One is that development skills are valued more than domain-knowledge or knowledge about the end-user. Creating a test-framework “from scratch” is over-hyped, fueled by a multitude of influencers and bloggers.
A reason that few like to talk about is that it is easier to write some code instead of thinking through what it takes to actually test a given piece of software. You would have seen cases where a team spends months building a test-framework, only for a new team to come along and promptly decide that a re-write is required.
To help teams make the right choice, we have created this graphic to emphasize a few points. It can be used as a handy reference to compare test-automation tools and discuss the relative importance of the capabilities for your specific team.
Please pass this on to those who are responsible for driving standards and evaluating tools in your enterprise.
To summarize:
- Test Automation requires a large set of fundamental capabilities. Are you sure you can implement all these in-house, or would you be better-off using an industry standard solution?
- Does the tool pay attention to Developer Experience? Keep in mind that tests must be maintained for as long as your systems are in production.
- Does the tool bring developers and QA closer together or force them into different user-interfaces and workflows?
- Can your tests be version-controlled just like your code? To be specific, can you see the history and the “diffs” between previous versions?
- Is your test data within your firewall or going into “somebody else’s cloud”?
- Is it easy to simulate end-user-flows and the core capabilities (your reason to exist) of your business?
- Is there a mature community of users or someone that you can reach out to for support?
- How useful is the documentation? Does it go into detail? Are there enough examples?
- How easy is it for someone new to the tool to start using it productively?
The list of capabilities in the graphic is a result of many iterations and discussions. Do let us know if any test-aspect is missing, or if you have any other feedback!
And by the way, the boxes in blue are things that Karate does today.
Happy Testing!
Author
Peter Thomas, Co-founder & CTO, Karate Labs
Peter is recognized as one of the world’s top experts in test automation. He brings 25 years of industry experience from which he has been in open source for the last 18 years. He has worked at Yahoo and Intuit. As part of the API platform leadership at Intuit, Peter created “Karate” the open-source solution unifying API, UI & Performance testing. Peter was one of only 15 chosen by GitHub for a grant in India 2021. He co-founded Karate Labs Inc in Nov’21 to accelerate the adoption of Karate with the mission of making test automation fun and collaborative. Karate Labs is a Y Combinator backed company.
Karate Labs is a Platinum Partner at EuroSTAR 2023. Join us at Antwerp Zoo June 13-16 in a 4 day celebration of testing. Learn from 65 expert speakers and connect with your peers at Europe’s Best Testing Event. Book your tickets by Dec 31 and save 20% or book your team and save up to 45%.