2023

10 Points to Help You Choose the Right Test Automation Tool

March 24, 2023 by Lauren Payne

Thanks to Testsigma for providing us with this blog post.

Making a decision to start test automation is easy but choosing a right test automation tool is not. There are teams that are spending a lot on hiring new manual testing resources but find it hard to invest in automation. The reasons could be many.

Sometimes, teams spend a lot of time exploring tools and get so overwhelmed with the information out there that they give up on the idea of automation altogether. Other times, choose a generic tool, start with automation but then, never get past the first few test cases.

In this article, we have put together some points that will help such teams navigate their search for the right test automation tool.

Points to Select the Right Test Automation Tool

1. Project Requirements:

There is no point in looking for a solution when you don’t know the problem. So, before you start exploring the various tools and technologies available in the market for test automation, you need to list down your project requirements and the problems you are looking to solve.

The list, in general, should answer the below questions:

Type of application that needs to be tested: It could be web, mobile, API or a desktop application.
Platforms that need to be tested: If yours is a desktop application, list down the operating systems that should be tested. If yours is a mobile application, then list down the supported mobile operating systems. If yours is a web application, then list down the supported browsers.
Language your application is built in: This can help if you are planning to use a programming language for automation.
Need for cross-browser testing/cross-device testing: If yours is a web application or a mobile application then you will, most probably, need this.

In addition, you could also add requirements that you deem important.

2. Team Skills / Learning Curve:

When selecting a tool for automation there could be 2 types of tools:

A codeless test automation tool
An automation tool that requires coding

If yours is a team that already has people that are skilled in some programming language then you can think of using an automation tool in that programming language. Or, if you plan to hire skilled people for automation then you don’t need to consider this point.

But, if you are planning to have an automation tool that will not need you to look for people with the required skillset, going for codeless automation tools will be a good idea. These tools allow the automation of test cases without the need for knowing a programming language.

Check this guide to know about Codeless Testing in detail.

3. Budget:

This is a very important aspect of choosing the correct automation tool. You might easily say that you will want a free tool because you don’t want to spend on automation if you can avoid it.

But, you also need to consider that the amount of time being spent on automation, the number of people working on the tool and the machines being used for automation also constitute the amount you spend on automation. So, consider below points before deciding the budget:

Cost of human resources being used for automation: If there is a tool that does not need you to hire new resources especially for automation, consider it a saving.
Time spent on learning the tool: If there is a tool that has a low learning curve, that is an indirect saving in the cost you might have spent in terms of the time the resources spend on learning the tool. Or hiring resources that are skilled in that particular tool.
Time being spent on automation: If there is a tool that makes it easy to create and maintain test cases, thereby saving time, consider it a saving in cost.
Cost of infrastructure: Talking about cloud and hosting, you can go for an ideal PHP hosting that gives an amazing managed hosting experience.

4. Ease of Test Case Creation and Maintenance:

Not every tool is made to handle all kinds of scenarios. So, to make sure that your chosen tool meets your needs, try automating a few test cases of your application to know if the tool suits your needs. That could be done with the trial version of a tool if your search has narrowed down to premium tools.

Also, to avoid spending more time in test case maintenance as compared to test case creation, make sure to choose a tool that fits your budget including the maintenance costs. There are tools that have the ability to self-heal the test cases in case of minor changes in the application.

Such tools help to reduce the cost of test case maintenance. Also, it helps if the tool supports pause and resume of test case execution for a better debugging experience.

5. Reusability:

To avoid writing the same code multiple times in multiple test cases and to avoid duplication of efforts, look for tools that allow the reuse of already created test steps in different test cases and projects.

6. Data-driven Testing:

If yours is an application that needs testing for a variety of data at multiple interfaces, it is important to choose a tool that supports data-driven testing.

7. Reporting:

Test case creation and test case execution would be useless if the reports were not useful so do go through all the features in the reporting supported by a tool. Few of them would be:

Screenshots for failed steps
Video for test execution
Stack trace for the error
A clear indication of failed test cases/steps
Time taken for execution of test steps and test cases is reported

8. Support for Collaboration:

If you are doing automation of a project for a client, the client will want to review the quality of automated test cases.

It will also be beneficial if other non-technical members of the team are able to automate/review the test cases. So, in such scenarios, look for tools that make collaboration with the management and clients easy.

9. Support for Tools for Integration:

If there are some process improvement or CI/CD tools that you already use or plan on using, make sure that you chose a tool that integrates with them.

10. Training and 24×7 Support:

Consider a scenario – you started using a tool for automation, and after automating about 10 test cases successfully, you got stuck on the 11th test case; you don’t know how to resolve the problem. You have looked at all possible forums but you don’t have a solution in sight. If you want to save the time, use a tool that has 24×7 support to resolve any problems you encounter.

Conclusion

Sometimes, teams decide to create their own test automation frameworks because they cannot find a right test automation tool that fit their testing requirements.

At the moment, there really are multiple types of test automation frameworks and tools available in the market that support automation on a varied variety of applications and are still being improved.

So, do go through the above points and spend some time looking at available test automation tools before thinking of implementing a framework on your own.

At the end, I would like you tell you about a test automation tool Testsigma. It is a cloud-based test automation tool that lets you automate test cases just in simple English – no coding required. In addition, you can automate your test cases for web as well as mobile apps at the same place. Do check it out if it meets your needs.

Frequently Asked Questions

How to Choose Automation Tool?
To choose the right test automation tool, you have to ensure it’s capable, powerful, flexible, and up to the mark for your project requirements. By saying capable, powerful, and flexible, your selected tool should be capable enough to manage all the test cases and test data smoothly. It should also be flexible enough to integrate with other third-party tools to extend and customize the functionality and make testing even easier.

Which Tool is Mostly Used for Automation Testing?
There is an abundance of testing tools available in the market that can be divided into two categories. The first one is the no code low code test automation tool which requires little or no knowledge of programming to perform any type of test automation.

What are the Criteria for Selecting a Test Automation Tool for Your Project?
Criteria to select a test automation tool are the following:

Capable- It should be able to manage the project test cases and test data efficiently.
Flexibility- It should be able to integrate with other third-party tools to extend the functionality tool.
Cost-effective- It should come into your project budget.
Learning curve- Learning to use the tool should not be challenging for your other team members.

Is Selenium the Best Testing Tool?
If you are from a developer background, then Selenium is a free and open-source tool project that provides various tools, resources, and libraries to make test automation easy for everyone.

When should We Choose Automation Testing?
When you required speed and accuracy at the same time, you should go for automation testing instead of manual testing, automation testing enables your team to do more in less time by providing the features like test cases to manage the project all test cases, test data management to manage the project test data, test labs to run the test case in the combinations of operating systems and browsers, etc.

Author

Shruti Sharma

Shruti is a writer and a content marketer with more than 10 years of experience in testing and test automation, and has been associated with Testsigma since about 3 years. She loves to read, learn, and write in detail about testing, test automation and tools. In addition, she also writes fiction. One cause she deeply cares about is mental health and psychology.

Testsigma is an EXPO Gold partner at EuroSTAR 2023, join us in Antwerp

Exploratory testing in agile

March 7, 2023 by Lauren Payne

Thanks to Xray for providing us with this blog post.

The purpose of exploratory testing is to learn how a particular area of your testing is working while using your skills as a tester to offer insightful input to your team. Through exploratory testing, you can ensure that all bugs are detected, and developers can fix them in time for the product release.

Exploratory testing is important and should be a component of your testing strategy since it helps you evaluate your tests’ efficacy, identifies code inconsistencies, and removes bottlenecks where defects are most likely to lurk.

In this post, our solution architect Sérgio Freire gives you the best tips on how to do exploratory testing in an agile environment.

Working in an Agile context

At the start of a product or one of its releases, we assume that we know everything there is to know about the product. However, there are often many unknowns and assumptions.

Agile comes as a way to deal with the complexity and unknowns around the whole software development and delivery process. When working in an Agile context, the software is delivered in small batches, known as iterations.

The idea is to reduce the batch of work and learn with small experiments. So, instead of working for a long time on a complex feature, we iterate on it by collecting feedback, learning, and incorporating our findings.

All this means that many changes are due to these iterations driven by our findings and feedback.

Exploratory Testing and Agile

The following model called the “Learning Lollipop model” (created by Sergio Freire), tries to highlight what happens during exploratory testing.

It’s a way to frame exploratory testing where we “taste” our product/product ideas (like tasting a lollipop), starting with questions that will give us ideas for designing testing experiments that we execute and then analyze. From this process, we learn. In turn, that will raise additional questions that will trigger new ideas for test experiments. While we do so, we walk in the unknown lake that contains all possible usages of our product. The more we explore, the more we find.

Using an example, let’s first try to see how these actually work well together. Say we are working in a system with a set of features, aiming to add a new feature using an Agile approach.

From a testing perspective, what we had (i.e., how the system behaved) should be covered by a set of checks, using test automation scripts as much as possible. This will allow us to collect almost immediate feedback from the CI pipeline(s).

We cannot simply retest everything from the past “by hand.” We also know that test automation scripts are fallible because they will always be limited to testing what they’re hardcoded to check; however, they give us a good starting point.

Whenever iterating on a new feature, we know that we don’t know much about it beforehand; that’s why we’re iterating it, after all. Usually, we’re dealing with a rough user story and not an extensive, highly detailed requirement.

Therefore, we need to test our initial ideas for the user story and depict areas/risks we should have in mind. Many questions will come, at the start, during, and after the implementation. All these can become ideas for test charters that we can explore with Exploratory Testing sessions.

Remember that in an Agile context, changes are frequent, and risks also change very dynamically.

Exploratory Testing is a great fit in Agile, as it is extremely flexible and doesn’t require upfront preparation (as happens with manual scripted test cases). It also uses information from previous sessions to drive new testing sessions. Therefore, it adapts to changes as it doesn’t assume a certain state and expected results for the system.

Tips for exploratory testing

Exploratory testing mockups.

Perform exploratory testing sessions on early mockups, internally and with users. This can be quite helpful to optimize flow problems, for example, and highlight the most valuable ones. You can also apply exploratory testing during your design sprints.

Discuss upfront with the team possible charters for your exploratory testing session.

During regular meetings (e.g., standups, planning), discuss with the team the test charters (i.e., the questions you aim to answer during testing). It’s a good moment to talk about risks and have insights from different team members, giving ideas for further exploration.

It’s always a good time to perform an exploratory testing session.

Whether you’re adopting waterfall or Agile, it’s always a good moment to perform some exploratory testing. We will never know everything about our product/system and its context, but we can further improve our understanding by conducting exploratory testing sessions. There are many quality attributes we can look at, for example. Consider aspects that concern your team, users, and business, and use that to drive new sessions. Taking some time to explore is investing in knowledge so that we can then work towards incorporating that feedback and improving our product/system.

Use exploratory testing to highlight ideas for test automation scripts.

Features should come with code, including unit and integration level tests and even system tests if appropriate. Whenever performing exploratory testing, one of the outputs can be ideas for test automation scripts. During exploratory testing, we may find flows, impacts, and edge cases, for example, that, due to their relevance, should be covered by “automated tests.”

This applies to the waterfall and Agile projects and will allow us to improve test coverage addressed by test automation and hopefully gain more time to focus on other tasks (e.g., further exploration, fixing problems, etc.).

Perform exploratory testing on the feature branches or the PRs.

If your team uses feature branches while features are being implemented, you can and should test. This means working with developers to improve the feature iteratively. You may perform an exploratory testing session around a certain risk, quality attribute, or subset of that feature at a given moment. You can also perform a session when the PR is ready for review; if you tested while it was implemented, then this moment will eventually be more of a high-level type of charter.

Perform exploratory testing after merging branches.

Merges sometimes produce unexpected results. Even though the feature branch may (should) include automated tests, there can be unexpected consequences, so scheduling an exploratory testing session can help uncover them.

Involve developers and other roles in exploratory testing.

Besides testers, getting others on board for exploratory testing can provide additional perspective. At the same time, foster a quality centric culture where team members can improve quality from the start in the future.

Pairing with a developer, with the PO, with a designer is a good practice to understand not just the system from different angles but also what different stakeholders expect from it; besides, it’s an excellent mid/long-term investment towards better quality.

Don’t limit exploratory testing to non-regression.

Even whenever automated regression tests may cover existing features, it’s a good practice to perform exploratory testing also for regression testing if you have the opportunity to. Test automation can cover the essentials, but we know many things escape these tests as they will always be limited in number and scope. Looking back at your previous features with your eyes wide open may depict problems added meanwhile and problems that you didn’t have the opportunity to uncover.

Exploratory test your test automation.

Look at your existing test automation and explore it to look for problems (it’s also code, isn’t it?). Look also for problems in scope, concurrency, and relevance. Look at your existing test automation logs, as they may provide valuable information or expose too much or too little information.

Exploratory test using tools to augment testing and gain efficiency.

Tools are used to perform certain tasks with efficiency and consistency. In exploratory testing, tools are used to augment the tester’s capabilities, not to replace the tester. An exploratory tester will easily use tools to facilitate API requests and assist with performance testing. With tools, an exploratory tester can be more efficient and cover quality aspects that otherwise would be hard or even impossible to tackle.

Exploratory test looking for gaps and opportunities to improve the value.

While testing, we look for problems that affect the quality and, therefore, the value as seen by different stakeholders. Testing is about understanding how the system works connected with expectations from all these different stakeholders. In this sense, testing is also about finding opportunities to increase the value. During exploratory testing, and using our knowledge and background, we can depict ways of improving the value of our products. Maybe that can be about framing the feature slightly differently, trying out a new form or interaction.

Bringing some agility with Exploratory Testing to waterfall projects

For organizations working on waterfall-based projects, testing mostly occurs after features have already been implemented. We know that if this happens, then the cost of fixing problems increases considerably.

Usually, there are initial requirements that drive implementation. These highly detailed specifications are not immune to problems; on the contrary: they can be built on top of many assumptions and lack actual user feedback.

We know that requirements, and specifications, in general, are incomplete, ambiguous, sometimes contradictory, and easily get outdated.

As exploratory testers, we can use not only requirements and other documents as the source for our tests; we also understand the context of our product, know about similar products, and know of known heuristics that can help us expose problems through test tracking and reporting. We also have our background that we can use to expose risks and impacts that otherwise could escape traditional testing.

In waterfall projects, we can use Exploratory Testing to help us:

Uncover problems, risks, and gaps that we couldn’t predict beforehand as they were not identified in the requirements/specifications.
Introduce testing while the feature is being implemented and thus refine it before it’s too late.
Complement traditional approaches, such as manual scripted test cases, with exploratory testing to go beyond the obvious and expose problems that we could otherwise miss.

Unleash your testing potential with Exploratory Testing

Exploratory testing promotes innovation instead of scripted testing that is centered around specified test cases and attempting to complete a fixed number of tests per day. Exploratory testing encourages us to act role play as the end-user and detects more realistic bugs.

Exploratory testing is highly helpful in agile environments and has several advantages, as seen above. QA teams can successfully use this testing strategy for their own success in the agile development process by knowing its benefits and using reliable test management software like the Xray Exploratory App.

Author

Sérgio Freire, Head of Solution Architecture & Testing Advocacy at Xray

Sergio Freire is a solution architect and testing advocate, working closely with many teams worldwide from distinct yet highly demanding sectors (Automotive, Health, and Telco among others) to help them achieve great, high-quality, testable products.

By understanding how organizations work, their needs, context and background, processes and quality can be improved, while development and testing can “merge” towards a common goal: provide the best product that stakeholders need.

Xray is an EXPO Platinum partner at EuroSTAR 2023, join us in Antwerp

Is test automation a first-class citizen of your development pipeline?

January 18, 2023 by Lauren Payne

Thanks to Karate Labs for providing us with this blog post.

A key point that many teams fail to ponder: “Is test-automation a first-class citizen of your development pipeline?”

This is the second part of a series of articles covering the finer aspects of test automation in detail. Read the first part here: The Test Automation Capability Map.

What happens when Developer Experience is not prioritized? Does this situation (see picture below) look familiar where you use a separate tool and workflow for authoring test-automation artifacts?

Let us zoom in on three of the Test-Automation capabilities which fall under the category of Developer Experience.

IDE Support

The development team spends the most time within their IDE of choice (e.g., IntelliJ, Visual Studio Code etc.). If test-automation requires a completely different tool, user-interface, and workflow, this has several implications for your team:

Switching between tools takes developers out of their “flow state.”
Developers are less likely to run tests before checking-in or merging code. This leads to an inefficient feedback loop, where failures are detected only when tests are run later.
Developers are less likely to contribute and maintain tests. This results in dysfunctional teams where developers “chuck things over the fence” to the “QA team”. It is very common to find developers and QA teams operating in silos where bugs that “escape” to production result in finger-pointing and blame-games.

Self-Hosted or On-Prem

An aspect often overlooked is whether any sensitive data is leaving your safe-zone and coming to rest beyond your organization’s security perimeter or firewall. If you use a SaaS tool that is not self-hosted, this typically is the case.

Even though integration and end-to-end testing environments should ideally use fake or synthetic data, there will be cases where “production-like” data will be needed to simulate real business-scenarios. Many teams extract a “cut” of production data for a staging environment with some data-masking or sanitizing applied. Equally or more important than test-data is test-configuration, such as database-secrets, authentication-tokens, and passwords.

Given that a lot of teams operate in the public-cloud even for pre-production environments (e.g., AWS, Azure and GCP) it is even more important that the greatest care is taken to protect not just your test-data, but the configuration and locations and URLs of test-servers and other infrastructure. So, the critical question you should ask – Are my tests being stored in somebody else’s cloud?

Version Control, History, and Diffs

Self-hosted or not, the situation worsens if your test-automation artifacts are being managed in a separate tool or repository. The implications of having tests in a separate tool and workflow are understated, but significant

You will lose the ability to eyeball the changes in your tests, side-by-side with the corresponding changes to code.
There is less pressure (or no “forcing function”) to add or edit tests when code-changes are made. This discipline can make the difference between a high-performing team or one that lacks confidence to ship as often as needed.
Tests are the best documentation of “what the system does.” When you cannot see the history of changes to tests side-by-side with the code commit history, you lose a valuable chunk of documentation, and you are left with an incomplete picture of how the software evolved.
Your Continuous Integration job becomes more complex at the point where tests must be run. Instead of getting code and tests in one atomic “checkout” or “git clone” operation, you are forced to perform an extra step to download the tests from somewhere. Keep in mind that you also need to ensure that the version of the tests corresponds to what is being tested.

In addition, if you are using a tool with a “no-code” UI to author tests, it is quite likely that you lose the basic ability to see diffs and history even just for your tests. Some tool vendors have ended up having to build version-control into their user-experience, re-inventing what comes naturally to teams that use Git to collaborate.

Some things are best expressed as code. At the end of the day, everyone agrees that code-diffs are a superior Developer Experience.

To summarize, for your test-automation to be a first-class citizen of your development pipeline you need to use a tool that integrates into the team’s IDE of choice and stays close to the code being tested. The result of that is shown below.

An interesting observation is that: this is exactly the developer experience you expect for unit-tests!

So which kind of team do you want to be in?
Happy Testing!

Author

Peter Thomas, Co-founder & CTO, Karate Labs

Peter is recognized as one of the world’s top experts in test automation. He brings 25 years of industry experience from which he has been in open source for the last 18 years. He has worked at Yahoo and Intuit. As part of the API platform leadership at Intuit, Peter created “Karate” the open-source solution unifying API, UI & Performance testing. Peter was one of only 15 chosen by GitHub for a grant in India 2021. He co-founded Karate Labs Inc in Nov’21 to accelerate the adoption of Karate with the mission of making test automation fun and collaborative. Karate Labs is a Y Combinator backed company.

Karate Labs is a Platinum Partner at EuroSTAR 2023. Join us at Antwerp Zoo June 13-16, in a 4 day celebration of testing. Learn from 68 expert speakers and connect with your peers at Europe’s Best Testing Event. Book your tickets by Jan 31 and save 15% or book your team and save up to 40%.

Get in-depth training at EuroSTAR tutorials

January 17, 2023 by Fiona Nic Dhonnacha

The EuroSTAR 2023 tutorials are here to provide in-depth training and cutting-edge learning for you and your team. Solve problems, get new ideas, and LOTS of actionable takeaways.

Our 11 half and full day tutorials cover leadership, test automation, collaboration, management, and lots more. Gain insights to re-energise your testing projects and make your business stronger.

Want to learn more about how biases affect your testing? Improve your critical thinking skills? Learn new concepts and testing tools? This is where it all happens.

Full Day Tutorials

Inspiring Quality: Becoming an Exceptional Quality Leader

Fiona Charles

What does it take to become a quality leader? Inspiring quality is about much more than getting people to follow you. Find out the skills and personal strengths it takes to become an exceptional leader, and where to focus your energy. This interactive workshop will allow you to practice solving leadership problems, assess your potential, learn how to become the leader you want to be, and bring benefits to all the projects you’re working on.

Collaborative Test Design

Wim Decoutere & Michaël Pilaeten

When people work together, the outcome is often more successful than doing it alone – so why do we keep designing tests alone? Traditional test design techniques seem to be created for solo application. In this workshop, you’ll design great tests with your peers. Learn different techniques, and experience for yourself how well these techniques suit your need. The best part? You can take home the results that come out of this workshop. Not just the techniques, but also the test cases: expand your test design tool cases with techniques that you can practice with the whole team when you get back to work.

Let’s Build a Continuous QA Strategy around CI/CD Pipeline

Szilard Szell

Building a test strategy in DevOps is not only about test automation, but how built-in quality can be achieved in all steps of the SW development, within and outside of your CI/CD pipeline. Join this interactive tutorial to learn how QA is extended to the left and to the right in DevOps, how to design your QA pipeline using simple card games, and what else you need to have a full test strategy. Throughout the exercises, you’ll build a full test strategy with your team for pre-selected scenarios and contexts in breakout sessions, followed by presentations and discussions.

view full day tutorials

Half day tutorials

Measurement, Metrics, and Magic Numbers

Michael Bolton

Bugs that could have been found in requirements gathering cost 1000 times more when found in production. Testing costs 40% of the total development effort. As a tester, you’ve probably seen someone pull these kinds of numbers out of a hat. Numbers can be magical, but can also be deceiving. We must apply critical thinking when it comes to numbers – in this workshop, you’ll learn strategies and skills to help you think critically and confidently about numbers. Develop ways to provide reasonable, helpful answers when people ask for metrics, KPIs or estimates, and learn how to recognise when metrics are dangerous.

Building Better Teams Through Play

Jenny Bramble

How do we release tension and form the strong bonds that a good team needs to be successful? As adults, we lose the sense of companionship and playfulness that we had during childhood. Let’s bring it back! In this tutorial you’ll learn how to build bonds within your team, by playing games targeted towards solving common issues with teams such as conflict issues, agile concepts, and lots more. You’ll walk away with a set of games to help you solve problems – and maybe a prize…

CSI Testing – A Simulation Game

Adam Matłacz

Asking good questions and critical thinking are essential skills in software. Join Adam Matlacz to become a true QA detective by using the CSI testing framework. Play a simulation game where you’ll be assigned a role within the team of ‘investigators’. You’ll have to gather information to solve the puzzle – the success lies in how good your critical thinking skills are, and how well you master the art of asking good questions. Walk away with the knowledge of good team cooperation, and the ability to filter the valuable pieces of information from the noise.

Roll the Marbles! How Good Architecture Helps us to Test Better

Alexandru Cusmaru & Ard Kramer

Are we really aware of the value of solid architecture on our testing? Unfortunately, opportunities to improve our architectural decision-making are rare. In this tutorial you get to design and build a marble run track using the SCRUM way of working. Learn how important good architecture is, and understand its consequences for testing and quality in your team. Last but not least, have fun together playing with marbles!

Soft Skills of Automation

Jenny Bramble

In this half day tutorial, learn what it takes to think automation first, so that you can develop a mindset that leads you to create good automation. Jenny will show you how to interrogate the manual testing that you do and shape it to automation. You’ll also discuss and create tests using pseudo-code, and discuss what kinds of results you can expect from your automation. When you leave this tutorial, you’ll have a good idea of what it takes to think automation first, what types of testing are appropriate for automation, and what information we can get from automation.

Sharpen the Axe! An Adventure Game about Test Automation

Anne Kramer

Join Anne Kramer to experience in a playful way how to deal with typical challenges in test automation, such as chaotic product backlogs, legacy test scripts, deteriorating coverage, and lack of time. You’ll be split into groups of 4 and practice together well-established test design methods such as decision table testing, model-based testing, BDD, and keyword-driven testing. You’ll gain an understanding of how success and failure strongly depend on the way your team works together.

Social Software Testing Approaches

Maaret Pyhäjärvi

Collaboration between team members has a significant impact on the team’s ability to deliver value efficiently. We are all well aware of the need for developers, testers, product owners, and other team members to collaborate. In this tutorial, you’ll learn how to improve collaboration skills in paired and grouped hands-on work, and learn how to foster collaboration in teams during remote work. Learn new approaches with a more social aspect in order to improve testing, and work better with developers.

Critical Thinking Skills for Testers

Michael Bolton

To be a great tester, you need to be a great critical thinker. The good news is that critical thinking is not just innate intelligence or a talent—it’s a learnable and improvable skill you can master. Critical thinking begins with five questions: Huh? Ready? Really? And? So? These questions kick-start your brain. In this hands-on workshop, you’ll practice and sharpen your thinking skills, and learn new ways to identify bugs.

view half day tutorials

EuroSTAR 2023 keynotes: 5 incredible talks

December 19, 2022 by Fiona Nic Dhonnacha

Our EuroSTAR 2023 keynote speakers are here to deliver fresh takes, cutting edge insights, and lots of lightbulb moments to spark new ways of thinking for your work and testing projects. Check them out below, and get ready for a burst of inspiration.

It only takes one genius

Isabel Evans

Would we make more progress in testing projects if we had some more geniuses on the team? How about one genius? Would one genius deliver or totally mess it up? Designing and building software is not a solitary occupation, and testing projects are hard. We work in groups because the problems and challenges are too complex for one person to resolve – no matter how clever they are. A a single genius, or even a team of geniuses, might fail badly. A surer route to success is to build a well-balanced, egoless team, where everyone contributes and is respected for their skills and viewpoints. Learn why designing and building software is not a solitary occupation, and how to build a team that succeeds.

see talk

Testing with purpose: How stories, values and kindness make testing better

Kristel Kruustük

Learn how to better test with purpose, and uplift the entire software testing community. Sound simple? Kristel founded her company Testlio to try to solve problems in the freelance marketplace, including bug-based pay, a lack of learning opportunities, and isolation. She learnt that stories, values, and kindness can be transformational to our work, and create a purpose. Stories of testers and testing help us to understand and identify gaps in the experience and fulfillment people encounter in testing. Aligning testing teams around a set of values lets trust and purpose grow. Kindness allows us to see people who need support, learning opportunities, and have concerns and considerations outside work. These three elements together can help us build purposeful testing. Get real-world examples of how you can unite testing and QA teams, and invigorate their work and integration with development

see talk

Whose test is it anyway?

Maaret Pyhäjärvi

There’s a question that people keep asking, expecting there is a simple recipe they just need to learn: how would you test this? When your job is to find some of what others may have missed, the responses vary greatly. Software development is a social activity, and the social context determines what work gets priority. In this talk, learn Maaret’s 10-step recipe in exploratory unit testing of a program created using computer-assisted software authorship. You will be testing programs and systems, and learning how to succeed with the right ingredients: social software testing, perspectives and oracles, and strategies and tactics. See why it’s not about testers, but testing – and sharing tests in teams across test levels through collaboration.

see talk

The cyber security of smart ‘adult’ toys – or lack of it!

Ken Munro and Jo Dalton

Did you know that smart adult toys are widely used in the military, as couples are often separated when posted overseas? These toys allow for an intimate relationship to continue, despite being physically separated. However, the poor cyber security of these toys expose the safety and security of military personnel. Some of these toys expose the exact GPS location of the user to anyone with some very basic hacking skills. The degree of privacy invasion resulting from poor security of these toys is quite shocking. The manufacturers of these devices are slowly improving their security – but not fast enough. Ken and Jo will be bringing a number of these toys to this session in order to demonstrate the problems with them. See why validating input and output is so critical, and why authentication isn’t the same as authorisation – and where it often goes wrong.

see talk

We’re off to rehab – communicating in an isolated & decoupled world

Anne-Marie Charrett

At work, remote first is the new norm, with meetings on Zoom and chatbots reminding us to WOL (Work out Loud) or TOL (Talk out Loud). But it goes further. Technology-wise, our microservices architecture and CI/CD infrastructure demand isolate and decouple our features into micro-stories separating us from our customers. Concepts such as Team Topologies encourage us to divide our teams into stream-aligned customer-facing, platform, or enabling teams. Our context has changed, and we must revisit how we work, communicate, test and build software. What was once a casual chat by the water cooler must be designed purposefully and included in formal meetings. Where you once relied on non-verbal cues, the written word requires more thought. In this talk, you’ll learn when and how you communicate is critical to the message being heard, and how to make collaboration intentional, so that your voice is heard.

see talk

Check out our full programme and join us June 13-16th for 4 days of learning, testing, and connecting with your peers.

The test automation capability map

December 14, 2022 by Lauren Payne

Thanks to Karate Labs for providing us with this blog post.

Test Automation is serious business. With teams under pressure to ship more often for competitive advantage, ensuring the best-quality end-user experience is key.

The choice of a test-automation tool can make the difference between a team that has high velocity – or one that lacks confidence to release.

We have seen many teams that have made the mistake of writing an in-house test-automation framework. I have personally experienced the pain of developers who have inherited tests in such a framework multiple times.

Unfortunately, teams continue to write their own framework for many reasons. One is that development skills are valued more than domain-knowledge or knowledge about the end-user. Creating a test-framework “from scratch” is over-hyped, fueled by a multitude of influencers and bloggers.

A reason that few like to talk about is that it is easier to write some code instead of thinking through what it takes to actually test a given piece of software. You would have seen cases where a team spends months building a test-framework, only for a new team to come along and promptly decide that a re-write is required.

To help teams make the right choice, we have created this graphic to emphasize a few points. It can be used as a handy reference to compare test-automation tools and discuss the relative importance of the capabilities for your specific team.

Please pass this on to those who are responsible for driving standards and evaluating tools in your enterprise.

To summarize:

Test Automation requires a large set of fundamental capabilities. Are you sure you can implement all these in-house, or would you be better-off using an industry standard solution?
Does the tool pay attention to Developer Experience? Keep in mind that tests must be maintained for as long as your systems are in production.
Does the tool bring developers and QA closer together or force them into different user-interfaces and workflows?
Can your tests be version-controlled just like your code? To be specific, can you see the history and the “diffs” between previous versions?
Is your test data within your firewall or going into “somebody else’s cloud”?
Is it easy to simulate end-user-flows and the core capabilities (your reason to exist) of your business?
Is there a mature community of users or someone that you can reach out to for support?
How useful is the documentation? Does it go into detail? Are there enough examples?
How easy is it for someone new to the tool to start using it productively?

The list of capabilities in the graphic is a result of many iterations and discussions. Do let us know if any test-aspect is missing, or if you have any other feedback!

And by the way, the boxes in blue are things that Karate does today.

Happy Testing!

Author

Peter Thomas, Co-founder & CTO, Karate Labs

Karate Labs is a Platinum Partner at EuroSTAR 2023. Join us at Antwerp Zoo June 13-16 in a 4 day celebration of testing. Learn from 65 expert speakers and connect with your peers at Europe’s Best Testing Event. Book your tickets by Dec 31 and save 20% or book your team and save up to 45%.

The EuroSTAR 2023 programme is out

November 21, 2022 by Fiona Nic Dhonnacha

What’s waiting for you inside the EuroSTAR 2023 programme? Get ready to be blown away by 60+ sessions from 65 expert speakers, all delivering insights and inspiration on essential testing topics.

A big thank you goes to the 2023 Programme Committee, chaired by Kristoffer Nordström. Together with Fiona Charles, Maria Kedemo, and Rob Lambert, they have put together a wonderful programme.

There’s a session for you at EuroSTAR no matter what your day-to-day involves. Jump into the energy of the best testing conference in Europe – live and in-person at the most incredible conference ZOO venue in Antwerp. Stunning architecture, summer sunlight, magnificent nature, and majestic animals… there’s no better place to re-invigorate your testing!

Check out the programme and come together with your community, 13-16 June 2023.

see programme

Visionary keynotes

Get inspiration straight from the testing icons making an impact in testing. Isabel Evans will show you how to build a team that succeeds. Kristel Kruustük will reveal her formula to unite testing and QA teams. Maaret Pyhäjärvi shares her 10-step recipe in exploratory unit testing. Ken Munro & Jo Dalton will talk about why validating input and output is so critical. Anne-Marie Charrett highlights why the how and when of your communication is critical to the message being heard.

11 fantastic tutorials

This year’s tutorials deep dive into test automation, collaborating, team building, leadership, management, and lots more. Learn design techniques, see how to create a continuous delivery pipeline, develop skills to deal confidently with difficult testing situations, and more.

45+ interactive track talks

Do you want to know why testers are the true rock stars of IT? How to build high-quality products? Make yourself future proof? Get the answers to these and LOTS more questions at our track talks. These are collaborative sessions with Q&As at the end of each talk – don’t miss out on brilliant insights from Iris Pinkster-O’Riordain, Brijesh Deb, Brendan Connolly, Jecelyn Yeen, and more.

This is your community

As well as all the amazing talks, at EuroSTAR you’ll connect with testers and QAs across all industries and from 50+ countries. Chat, network, relax, share ideas, and play games with your peers – the Huddle Community Hub is the heart of EuroSTAR, and THE place to hang out! It’s also where the Test Clinic experts will help solve your testing problems, and you can enjoy hands-on challenges at the Test Lab.

Check out the programme, and join your community next June – we currently have 15% off full price tickets with our programme launch offer.

Tips on writing a great conference proposal

July 27, 2022 by Fiona Nic Dhonnacha

The submissions are flying in to this year’s EuroSTAR Call for Speakers, and the committee is so excited to read everyone’s great ideas. If you’d like to share your story on the EuroSTAR stage, and you’re still working on your submission, we can help. We asked the 2023 committee if they had any tips on crafting a great conference proposal, and get your submission noticed. Of course, they had lots of great advice to offer.

Check out some top tips below from committee members Kristoffer Nordström, Rob Lambert, Maria Kedemo, and Fiona Charles. If you’re thinking about submitting to EuroSTAR – or any conference – these tips will help you make the most of your submission.

start eurostar submission

Kristoffer Nordström

EuroSTAR 2023 programme chair Kristoffer Nordström created this year’s conference theme, Software Development is a Social Activity, and you can read more about the theme and its topics on our Call for Speakers page. One of his top tips is: “Everyone has a story that they can tell. Write a submission based on what you’ve learnt and what you’ve experienced from it.”

Fiona Charles

“Remember that, besides being a source of information, your abstract is a sales tool. You’re selling your talk idea to the conference organizers. If it’s accepted, the conference will use your abstract to help sell the conference and then to attract an audience to your session.”

Maria Kedemo

“Don’t be afraid to ask for help. I always ask some of my peers if they have time to review my paper. I ask for feedback regarding clarity, and if they would attend based on the paper and language, since I am not a native English speaker. I always make sure at least one of my peers has a different perspective than I do.”

Rob Lambert

“Choose a core purpose of the talk, and try not to deviate too much from that. Good communication always has a solid purpose – if you deviate too far from this in the submission, and the talk, and it could get confusing. Read the guidelines for submitting carefully, then read them again, and use the guidelines to structure your submission Keep a copy of your submission safe.”

Bonus tip! Read Rob Lambert’s Blazingly Simple Guide to Submitting to Conferences – in particular the section on making sense of a conference theme, generating topic ideas, and writing a good proposal.

Now you’re ready to submit an awesome proposal! If you’d like to speak on Europe’s largest testing stage, the EuroSTAR Call for Speakers closes 9th September.

start your submission