Fiona Nic Dhonnacha

If you’re new to testing or looking to progress to the next level in your career, we have lots of sessions on the EuroSTAR programme to help you do just that! Skyrocket your testing skills and make lots of valuable connections. Learn how to shape your testing with the best test design techniques, improve your testing abilities by uncovering your biases, discover powerful tools to help you with day-to-day tasks, and LOTS more.

Our programme committee chair Graham Freeburn has created a mind map so you can check out the whole programme and filter by topics, roles, and more so that you can find the talks best suited to what you’re looking for – check it out and start planning your EuroSTAR 2022 diary.

Test design with data combination testing and classification trees | Rik Marselis

Test case design is one of the core competences of the quality engineering & testing profession. If you want to properly shape your testing, which test design technique(s) do you use? And is it effective and efficient? What coverage can you achieve? Rik explores the Data Combination Test design technique, which uses Classification Trees.

This technique can be combined with 3 different coverage types, so you have a great way to align with the risk level of the application. Join Rik’s tutorial on DCT & CT and start applying his techniques, the very next day you return to work.

The good, the bad and the biased | Emma Lilliestam and Hanna Schlander

How does the brain work? Why do biases exist, and what are the pros and cons of biases? In this workshop you’ll learn about two thinking systems in the brain, and four categories of biases. We are all biased. Biases affect our everyday lives, both at work and at home. By learning more about them we can expand our view, shape our minds, and improve our testing abilities!

You’ll get the chance to deep dive into biases in groups, and discuss a selection of your favorite biases, before presenting your findings to the group.

Drawing for effective testing and learning | Davrondzhon Gafurov

You may have heard of the term ‘a picture paints a thousand words’, but what about ‘a drawing replaces 100 (textual) user stories’? Davrondzhon shares how you can improve quality assurance by using graphical representations.

Drawings can help you identify tests which are difficult to extract from (textual) user stories, especially negative test cases. Furthermore, drawings can be used not only for quality assurance purposes, but also as a tool for learning (for newcomers) and collaboration (to get a common understanding between various roles in team).

Toolsmithing – a live coding session crafting test tools on stage | Kristoffer Nordström

Even though testing is a highly intellectual pursuit, it still involves a lot of manual steps, laborious work, and tasks to perform. How many times have you had to unnecessarily download a bunch of files from a remote test server after running your tests, or download them via a browser, clicking on the links to the files and saving them?

In this live coding session, Kristoff used Python to show you short, powerful, and useful examples of small test tools, that address specific day-to-day tasks, of the same kind as the ones described above.  At the end of the day you will have seen several examples of how you can use Python when crafting your own test tools.

21^st century skills for a tester | Emna Ayadi and Ard Kramer

What are the skills that you need to survive the digital 21st century? Soft Skills! In this tutorial, Emna and Ard explore the skills a 21st century tester needs. This can be defined with 4 Cs; Communication, Critical Thinking, Creativity, and Collaboration.

How do those 4 C’s relate to testing? Are testers already applying those skills, or have they thought about how to apply them in the future? Learn how you can use them to communicate, how they apply critical thinking in their organization, how they collaborate with team members, and how they apply critical thinking as a tester.

Drawing lines around the fog: the shape of testers, teams, and the world | Alex Schladebeck

The world is constantly changing, and everything is impermanent. After the last two years, we have been forced to come to terms with how quickly and drastically things can change. How will external changes shape our teams and our work? How can we shape ourselves proactively in order to be able to respond to changes, make changes or our own and even thrive?

Alex looks at what factors are at work now, and what kinds of effects will they have on how we work, and the roles of testers and software professionals. She will also look at concrete activities on an individual and company level, to best prepare ourselves for a nebulous future.

Have you booked your conference tickets yet? It’s not too late to join us next week – book your ticket now and we’ll see you for a celebration of testing in Copenhagen.

Thanks to Eduardo Amaral, Quality Management Director at Noesis, for providing us this blog:

When I hear someone talking about test automation, its benefits, and the investment needed, I always remember an example I’ve recently witnessed: 50% of the regression tests were executed in an automated process for more than a year on a specific eCommerce app from a well-known company. This company had one Automation Engineer working full time on the project to achieve it.

The project also had a quality engineer to execute the remaining 50% of the regression tests needed manually, and all new features. These new features testing represented around 50% of the global execution, which means that the Automation Engineer (a more expensive resource for the project) was only handling 25% of tests delivered. The remaining 75% was executed manually!

It’s a fact that automation is needed and beneficial to the software development lifecycle. It is the only way to keep high-quality assurance standards in a fast-moving market. So, the 1 million dollar question is: What’s missing? Why are most of the companies failing in this automation effort? Why are they just replacing manual testing with automated testing without any efficiency gain?

On the contrary, if we consider that the development costs are higher than manual testing, the reality is that those companies are just increasing costs rather than reducing them (the primary goal of test automation).

There’s no correct answer or a magic solution to make automation happen and work, like in all other foundational changes. Because automation is such a trend and hype in the industry, many companies are just doing it because of it, without having a real need or a structured business case. And that is not a detail. Moving forward with a technical proof-of-concept instead of a proof-of-value is the wrong approach and reveals a lack of strategy!

Achieving success in automation strategy

This lack of suitable strategy regarding the company’s organization and needs, its application landscape, and business processes, makes the adoption of standard automation processes the obvious path – it seems so obvious that failure is not even considered. “If I execute these tests manually today, tomorrow I’ll hire an Automation Engineer and run them in an automated way without additional effort!”

Where can this line of thought fail? It’s simple: we only execute repeatedly (the justification for automation) if we are talking about an application that is constantly being upgraded. Those are the apps that require intense automation maintenance, which also means breakeven slippage and ROI underachievement. And it gets even worse when a manual tester, especially skilled in focussing on business processes and setting up quality assurance strategies according to risk analysis and error patterns, is replaced by an Automation Engineer focused on technology and test patterns.

To achieve success in any Automation strategy, it’s vital to coexist and optimize (not replace) these two profiles – manual tester and automation engineer – during the development lifecycle, taking advantage of what each approach can offer. And most of all, it’s necessary to understand the application’s characteristics and adopt the best automation strategy.

When we’re talking about solutions where time-to-market is crucial, it is impossible to succeed without a seamless continuous testing strategy, where test and validation happen on the continuous integration pipeline. On the other hand, evaluating complex application architectures with significant dependency levels and integration with other systems, such as ERP, Billing, HR, Legacy, etc., makes it difficult to have an efficient use case for automation. The focus must be E2E and only use automation on specific and routine quality assurance processes instead of trying to automate all testing frameworks.

Test automation will be an essential part of any Quality Management strategy in the future and a complement for quality assurance, that’s certain.

But, to have the ambition to automate all the process for every application developed is a dangerous path, with a great probability of not achieving the desired results and producing a financial loss.

Noesis is a Gold Partner at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now.

Author

Eduardo Amaral, Quality Management Director at Noesis

Eduardo has worked as an analyst/developer, project manager, program manager, and service delivery manager. He has a background and certifications in several domains, including quality assurance, project/program management, process management, and product management. Over the last 21+ years, he’s been involved in a variety of software project development lifecycles and solutions integration, at high level of complexity and customer experience exposure. He’s performed in some of the major national and international companies, from different business areas – industry, banking, insurance, telco, retail, etc.

EuroSTAR 2022 kicks off in just over a week, and it’s going to be our biggest year yet! We can’t wait to welcome software testers from near and far to Copenhagen for our 30^th EuroSTAR Conference.

Have you got your ticket yet? We have a limited amount of tickets left so book now, and secure your place at the best testing event of the summer! Here’s what you can expect…

Star keynotes

Our keynote speakers are driving testing forward with innovations – Smita Mishra talks about testing software for the next economy; Rikard Edgren shows you how to generate a good understanding of testing through continuous learning; Michael Bolton shares insights on testing through digital transformation; and Alex Schladebeck talks about the future of testing, and how it will shape teams and testers’ work.

Expert tutorials

This year we have 10 interactive tutorials on all topics from agile to strategy, new technologies, DevOps and lots more. Our speakers will share learnings that you can apply into immediate and actionable tactics, both personally and professionally.

Tutorial trainers this year include James Lyndsay, Rik Marselis, Anne Kramer, Gáspár Nagy, Emma Lilliestam, & Hanna Schlander.

Track talks

55 track talks means plenty of opportunity to join sessions that motivate your passions, whether you’re a new or experienced tester. Track talks cover everything from testing techniques to management, skills development, challenges and lots more… talks are followed by live Q&As so you can get your tough questions answered in real time.

Track talk speakers include Dorothy Graham, Michaël Pilaeten, Ard Kramer, Andrea Jensen, Kristoffer Nordström, and Isabel Evans.

Huddle Community Area

EuroSTAR is all about community – and Huddle is at the heart of it all. Take part in speaker couch sessions, community soapbox, lively conversations, fun games, and impromptu chats in the Huddle area. Talk to our Test Doctors at the Test Clinic to get even your toughest testing problems answered, try some hands-on testing at our Test Lab, and so much more.

Networking Events

Our evening events are extra special this year as we celebrate 30 years of EuroSTAR – join us on Wednesday night for the EXPO Networking Party: drinks, chats, and a chance to meet some new people. On Thursday night it’s the Gala Dinner Night, where we celebrate with a night of great food and a show at Wallman’s, followed by the EuroSTAR Software Testing Awards presentation. We can’t wait!

… and much more!

Of course, there’s also lots of spontaneous meetings that result in lifelong friendships – make new friends over coffee, lunch breaks, and in between sessions. You’ve also got access to the largest software testing expo in Europe, so you can check out the latest software testing tools and be in with a chance to win some cool swag and prizes!

Thanks to Raj Subrameyer for providing us this blog post on behalf of TestRail:

Organizations are striving toward releasing features faster than ever, to meet growing customer demands and stay relevant in the industry. The market is highly competitive, and teams are looking into lean approaches to deliver high-quality products. Various agile approaches have been adopted to support this effort, and one of them that has gained popularity in recent years is DevOps.

What is DevOps?

Historically, development and testing efforts were viewed as two separate entities. This model proved to have several bottlenecks. Some of which included:

• Gaps in communication between the groups, leading to defect
• Incorrect understanding of the implemented features
• Not having clarity on test coverage
• No visibility into each other’s work
• Unnecessary friction due to lack of collaboration
• Unstable features pushed to QA

And most of all, no one had any idea of how the customers were using the deployed product. They did not communicate directly with the operations team monitoring the application in production, and vital statistics that could further improve the product weren’t gathered. This wasted incalculable amounts of time, effort, and cost for organizations and ultimately led to a delay in releasing the features on time and resulted in a decline in customer satisfaction.

DevOps was introduced to mitigate the above problems and reintroduce the mentality that quality is everyone’s responsibility. The main focus of this approach is to increase collaboration between the developer, tester, and operations teams, so they can work as one single unit to deliver high-quality products.

Role of QA in DevOps

In other agile approaches, the developer is in charge of coding and creating the feature and writing and running unit tests; testers perform automated and manual tests on this feature. DevOps purposefully blurs the lines between these responsibilities, forcing collaboration. The developers start building the mindset of continually checking for errors in their code. The testers increase their responsibilities from validating the application, to ensuring it is deployable at all times. They may even fix code as needed. All these pieces work together to ensure rapid delivery of features. The focus on the customer drives the work of the entire team.

In DevOps, the testing perspective changes from “completed testing on this module” to “what are the business risks that have been mitigated in the release candidate?” They move beyond manual testing and focus on scalable deployments and automation right from the beginning of the development process, standardizing QA environments, and aligning QA tasks with Continuous Testing, and CI/CD pipelines.

A high level of collaboration between developers, testers, and the operations team enables testers to monitor the application in production and use the data to help stakeholders make informed decisions.

How to integrate QA into a DevOps environment?

DevOps does not replace the existing QA process. Instead, it shifts testers’ viewpoint from the traditional way of testing, such as mapping test cases to requirements, executing test cases manually, and reporting defects; to a more agile way of delivering software, where QA is part of a large team instead of being a separate entity. Here are some ways to integrate QA into the DevOps environment:

Re-align team structure

First, organizations should remove the boundaries between developers, QA, and operations teams. This would involve restructuring the existing QA team to align with the DevOps goals.

Teams have to merge into one technical team that focuses on various activities to deliver the features on time. These technical teams have to communicate, collaborate and optimize their efforts to find defects fast and fix them before deployment. Also, testers should get involved early in the development process, starting at the requirements phase (where they can flesh out any ambiguity in requirements) by ensuring the whole team has clarity on how the feature will be developed, tested, deployed, and monitored post-launch. Continuous testing becomes part of everyone’s responsibility.

Automate as much as possible

Secondly, DevOps and automation go hand-in-hand—without one, the other won’t work. This is where smart people and tools can help in bringing shorter and more dependable release cycles. Everything that can be automated should be automated. This may include unit, deployment, integration, smoke, regression, security, and performance tests. Testers play an integral part in the DevOps process by starting automation early in the process, which finds defects sooner rather than later.

Also, for automation to be effective, you should modularize your test suites and run them at various phases of the development process based on the features that have to be tested. Finally, as new features are added to the product, the test suites should be flexible enough to scale according to complexity. This may include the need for additional software, hardware, and skilled resources to support the effort.

Use metrics to track progress

Another important factor determining the success of QA integration into the DevOps process is having metrics that can quantify the value. A survey conducted by Forrester Research found 75 common metrics that DevOps teams use to measure the success of the adoption. Some of them that directly relate to the quality of the products are as follows:

• Test case coverage
• Unit test coverage
• Pass/fail rate
• Number of tests executed
• API pass/fail rate
• Requirements covered by tests
• Blocked test cases
• Percentage of automated test cases
• Successful code builds
• New defects
• Critical defects
• Automated tests prioritized by risks
• Code coverage
• Release deadlines
• Total number of defects discovered in test
• New API defects found

Using these metrics, stakeholders can make informed decisions about how QA can effectively collaborate with the developers, operations team, and other roles in the project.

Run tests in parallel

Testers run tests in parallel to various activities in the DevOps process. For parallel testing to be effective, QA engineers need to set up standardized test environments and have sufficient hardware and software resources to run many tests as fast as possible. They may also have periodic audits to ensure the testing process is optimized to finish within the software delivery time.

Continuous monitoring

DevOps involves multiple tools, processes, automated tests, and environments. Data is extracted from different stages of the development process and analyzed to ensure the system is performing at the optimum level at all times. Usually, the operations team works with developers and testers to build monitoring and analytics capabilities right into the developed application. It is vital for testers to know how to use tools like Splunk, Kibana, Graphite and Firebase that are used for this purpose to support the operations team.

Have a common set of processes and tools

One of the biggest problems in organizations is each team follows a different process and uses a distinct set of tools to accomplish the same task. When this goes unnoticed, it leads to unnecessary wastage of cost, time, and effort for the organization.

With DevOps, the teams collaboratively have to agree upon a common set of processes and tools everyone will follow. Once decided, each team member will know what process to follow and tools to use at any given time in the development process.

Continuous feedback

Organizations must emphasize the importance of receiving continuous feedback from the team during the implementation. Daily standups, team meetings, retrospective meetings, and planning meetings are great examples of touchpoints to gather feedback. Based on this, teams will modify their approach for the next iteration. This continuous cycle of build->measure->learn (validated learning) helps to improve the DevOps process.

Increase visibility into tasks and processes

For a successful DevOps implementation, teams have to collaborate and have visibility into each other’s work. At any instant in time, teams should be able to visually see the progress of the development process. Organizations can use tools and dashboards that highlight vital metrics such as team velocity, defects logged/fixed, number of tests executed, test coverage, and other KPIs important to the team.

Sufficient training

Testers’ skills need to adapt to the DevOps process. It starts by training everyone to do full-stack test automation, which includes writing automated tests such as unit tests, API tests, UI tests, and back-end tests. Manual testing will still be relevant, but the focus will shift from only executing test cases manually to doing more risk-based testing and exploratory testing. This is performed in parallel to automated tests.

Test strategy and test design become even more critical in the DevOps process. Since testers are working along with developers and operation teams, their test strategy will have to include tasks related to QA and areas that cut across development and the operations teams. A good approach to come up with a solid test strategy and design tests is for different roles to work together collaboratively. This helps the team gain a common understanding of what needs to be tested, who will be performing them, and how the tests will be executed.

Finally, to successfully transition into DevOps, organizations should implement three essential processes: continuous testing, integration, and delivery.

The three pillars of DevOps

The main goal of DevOps is to increase collaboration. This is not achievable without having Continuous Testing, Continuous Integration, and Continuous Delivery. Testers play a vital part in each of these processe

1. Continuous testing (CT)

CT is all about testing an application from its early stage and automating tests throughout the lifecycle of the development process. This helps the team to analyze the quality of the product at every stage of the process and enables stakeholders to make informed decisions.

CT also reduces the risk of shipping features that break in production, saving a considerable amount of time, cost, and effort for the company. The key is to test as early as possible and find defects as soon as possible. Here are some things to consider when integrating CT with your current testing process:

a. Have sufficient resources: If testing has to be performed in each stage of the development process, the teams need to have sufficient resources: hardware, software, and skilled engineers. The team collaboratively decides on a test strategy and procures the necessary resources before implementing CT.

b. Align with the Shift-Left paradigm: CT aligns with Shift-Left testing, focusing on starting testing early right from the requirements phase and continuing throughout the development and release process. A part of this alignment is to expose developers and testers to different activities to empower them. For example, a developer would pay more attention to writing better unit tests, and testers can get involved in development activities. This helps to speed up the development cycles and builds cross-functional teams that can adapt to any environment.

c. Identify gaps in the existing test process: Before implementing CT, do an audit of your current test strategy, get a handle on your current test coverage, and the type of testing performed – manual scripted test execution, automated, and exploratory tests. Identify your current bottlenecks, analyze trends in production bugs and figure out how you can mitigate them with CT.

d. Use automation wisely: One of the common problems teams face when implementing CT is trying to automate everything, even if it does not provide value and could prove to be a maintenance nightmare in the long run. For example, automating features with a high state of flux could result in inefficient use of resources as they may spend more time maintaining the test than running them. Use automation when repeating mundane tasks, getting faster feedback on builds, deploying builds to different environments, and performing various kinds of testing that would otherwise be hard to do or consume time when done manually.

e. Have the right automation tools: For effective implementation of CT, teams have to invest in using the right tools for various activities. This could include tools for unit testing, UI testing, API testing, performance testing, security testing, and accessibility testing.

2. Continuous Integration (CI)

CI is a primary DevOps best practice of automating the integration of code changes into a central repository where it builds and runs several times a day. One of the key benefits of implementing CI is detecting errors quickly and locating them more easily. There are various benefits for QA in implementing a CI pipeline:

a. Remove dependency on code check-ins: Any number of people can check in code multiple times a day into the version control system. There is no dependency, and each check-in has timestamps to know who wrote the code exactly and pushed it to a particular branch. This means testers and developers can both write application and automation code and check it into different branches without any interference.

b. Faster feedback: Once the code is checked in, tests automatically run to get quick feedback on the build, so errors can be found quickly and fixed as soon as possible. It enables testers to quickly figure out what the problem is, report it, and take the necessary action to address and fix the defect.

c. Increased collaboration: CI improves overall engineering communication and accountability, enabling greater collaboration between development, testing, and operations in a DevOps team. Pull requests are tied into the CI process, helping to gain visibility into code from other team members. This enables developers and testers to collaborate on feature branches as the feature progresses through the CI pipeline. The automated tests have to follow the same pull request process.

d. Automated regression testing: As part of the CI process, testers configure regression tests to run when the build is deployed to different environments automatically. This ensures the new changes do not break existing functionalities. These tests are set up to run as quickly as possible to get faster feedback on the deployment.

e. Extensive documentation: When automated tests run in every stage of the CI pipeline, extensive logs are generated with timestamps for future reference. This is especially useful for organizations that have to adhere to regulatory compliance. The logs serve as a rich source of information during routine audits to ensure there is no breach of policies and procedures.

3. Continuous Delivery (CD)

One of the integral parts of DevOps is to deliver features rapidly. CD enables this to happen by giving teams the ability to release the application at any instant in time. Beyond making sure your application passes automated tests, it has to have all the configuration necessary to push it into production. CD encompasses everything from ideation to build to readiness to delivery to production.

The role of QA in a CD pipeline is to ensure continuous testing happens throughout the development process. Testers use various tools for smoke testing, regression testing, API testing, UI testing, performance, and security testing. Tools free up their time to do exploratory testing and risk-based testing to mitigate identified risks. Also, having a CI environment ensures automated checks happen in various phases of the build process, giving faster feedback on the quality of the features.

Key tools needed for DevOps implementation

Since DevOps includes continuous testing, integration, deployment, and delivery, many tools have to be used to get quick feedback about the application. Below are some commonly used tools in a DevOps pipeline:

• Source code repository: Git, Assembla, CloudForce, TFS, Subversion
• Build server: SonarQube, Jenkins, Artifactory, Travis CI, Circle CI, Kubernetes, Docker
• Feature flags: Rollout, LaunchDarkly
• Unit testing: JUnit, Nunit, RSpect, TestNG, Jasmine, Mocha
• Configuration management: Puppet, Ansible, Salt, Chef
• Test automation: Selenium, Appium, Watir, Ranorex, Cypress, Espresso
• Test management: TestRail, Jira, Zephyr, Xray, PractiTest
• Monitoring tools: Splunk, Kibana, Graphite, Crashlytics
• Virtual infrastructure: Amazon Web Services, Microsoft Azure, VMware vCloud
• Non-functional security tests – Mittn, Gauntlt, BDD Security
• Static analysis security testing tools – Kiuwan, Coverity, Checkmarx, and other paid tools. Open-source options include Reshift, Brakeman, JSHint, NodeJsScan.
• Security Scanners – Netspaker, Acunetix, Wireshark, Nessus, OWASP ZAP (Open Source)

As you can see, various tools can be used for different purposes in the DevOps pipeline. A good approach to selecting the best tools that work in your project context is by keeping the below factors in mind:

Choose tools that increase collaboration

Almost every tool used in the DevOps pipeline should facilitate effective collaboration between different team members. For example – Version control is a basic necessity in any agile project, including one that follows DevOps. The tool used for this purpose should help in the seamless integration of code, configuration files, scripts, and other artifacts across teams and different systems.

Tools should have open APIs

As technology advances rapidly, new tools come into the picture to solve existing problems. So, teams tend to shift to these tools. Migrating to new frameworks and tools should be assumed normal in teams following a DevOps process.

It is important to use tools that have open APIs so that even if the team migrates to a newer tool, teams can still use the existing scripts by calling the APIs with minimal disruption. Don’t get stuck with a single tool; instead, think about how easily any tool you choose can integrate with other tools through the use of APIs.

Ease of use

A DevOps team consists of people in different roles, including developers, testers, operations teams, business analysts, product managers, project managers, and other stakeholders. Any tool you choose should be able to be used by everyone without a steep learning curve. If a tool becomes too tricky to learn, people get intimidated and stop using them. When collaboration is the key aspect of DevOps it is critical to pay attention to how easy it is to use by people of various technical and domain backgrounds.

QA is an integral part of implementing a robust DevOps strategy. Faster releases and delivery cycles will not happen without a comprehensive QA-DevOps transition plan. The role of QA teams has shifted in the world of DevOps. Instead of individual contributors, they have become QA consultants. Testers will have to shift their mindset from providing traditional testing services to helping deliver quality software by mitigating risks across roles, including the development and operations team. They have more opportunities to teach people about the importance of quality as they will work closely with different roles. There will be no longer a separate QA team; instead, there will be technical teams that combine different roles.

QA teams will be forced to look for new ways to add value which will help to bring out their critical thinking skills. This could mean taking ownership and making some strategic decisions that they may not have done when working in traditional QA teams. There will be more focus on thinking of the software from an end-user perspective and how customers would interact with it daily.

Transitioning to DevOps is not easy, even if you’re already agile. One of the biggest obstacles to DevOps implementation is the process itself. It involves shifting the attitude of not only QA but all roles in a team. This takes a considerable amount of effort, higher standards, and more tools and resources. However, when teams do collaborate and have the right tools in place, organizations can start seeing immediate benefits.

How can TestRail help you optimize testing efforts within your organization? Join our introductory product demo to see how you can leverage TestRail’s core functionalities to improve your software testing processes, increase transparency and collaboration, and build more confidence in releases. Register Now!

TestRail is a Gold Partner at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now.

Author

Raj Subrameyer

Raj Subrameyer is an international keynote speaker, writer, and tech career coach with a rich technical background. In his blog, he posts inspirational news, resources, and updates to help his readers lead a better life.

Thanks to Sune Engsig from Leapwork for providing us with this blog post:

How can you get the most out of test automation? While it’s a cornerstone in DevOps, it doesn’t always go to plan. But when implemented correctly, it helps increase the quality of software while containing costs.

Not surprisingly, IT departments everywhere are realizing the importance of having a clearly defined test automation strategy, instead of taking a risk-based approach to testing.

Once the decision has been made to roll out test automation, the next issue presents itself: “How are we actually going to do this? What’s the plan?”

Whether it is regression testing, functional testing, or unit testing, having an overview of your test suite before you get started with a testing tool will help your team considerably.

Where to begin your strategy development?

Putting together a test automation strategy can seem overwhelming. It’s tempting to get stuck right in and put test automation to work. However, for test automation to succeed, it demands careful consideration of your team’s requirements, the time you have available, and the successes and failures of past projects – if any.

For example, if you’re coming from a manual testing background, testing will be slow. This causes QA bottlenecks and results in more human errors and bugs in production. Naturally, your requirement would be to adopt automation that will speed up development, and make availability to increase the scope of testing.

If you’ve used an automation solution that is highly complex, then it could only be used by a few technical test engineers. This prevents those who actually have domain knowledge from using automation. This would mean the requirements for your next project would be a tool that can be used and fully scaled across teams regardless of their technical capabilities.

How to build a test automation strategy

We have created a checklist of the areas to focus on – no matter whether you want to move away from manual testing or optimize your current automation process

You can access the full checklist to get a detailed description of how to create an agile test automation strategy. You’ll be equipped to develop a plan of action to achieve your test automation goals.

Leapwork is a Gold Partner at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now.

Author

Sune Engsig, Chief Evangelist at Leapwork

As Leapwork’s Chief Evangelist, Sune Engsig helps hundreds of organizations deliver business capabilities faster and more efficiently with automation. He has more than 20 years of experience working with business and process development as well as IT architecture for enterprises. In addition, he has extensive project management experience from both government and the telecoms industry. Sune is driven by a mission to find ways for businesses to extend their capabilities with the help of technology.

If you’re a test manager or team lead, we have lots of sessions on the EuroSTAR programme to help you elevate your team and achieve your goals. Join talks and tutorials led by expert speakers and learn how to develop your critical thinking skills and mitigate biases, scale your team successfully, understand the changing role of the test manager, and lots more. Our programme committee chair Graham Freeburn has created a mind map so you can check out the whole programme and filter by topics, roles, and more so that you can find the talks best suited to what you’re looking for – check it out and start planning your EuroSTAR 2022 diary.

Developing critical thinking skills for testers | Andrew Brown

Critical thinking skills are essential to testing – but rarely taught. Even the most seasoned tester is vulnerable to biases and thinking traps. In this workshop, Andrew shares the training course he’s developed in his organisation, to show you how to improve your testing by enhancing your critical thinking skills. You’ll explore thinking traps and biases, and learn how to mitigate your vulnerabilities. Take part in exercises and experiments throughout the tutorial, in order to learn more about how your mind works, and how you can think more critically.

Shaping a fast growing team like a bonsai | Seema Prabhu

As your team scales, the nature of challenges will change, Just like a bonsai tree, you can’t leave a team’s growth and shape to chance without a plan. One of the major components that a bonsai represents is that of balance. How can you achieve the right balance in a remote, multicultural team, and achieve your purpose ? Seema Prabhu shares her experience with her team changing shape as they scaled, the lessons learnt, and tools she wished she had during this phase of growth.

Testing software for the next economy | Smita Mishra

Smita Mishra shares what she’s learnt about the next economy, and what technology changes we are seeing across users. How does this impact what testers do? Do we understand the users of these new technologies? Are we ready to meet the needs of our customers of the next technology? Get answers to these questions, and a whole lot more. Let’s learn what the world thinks could be technology risks, and how testers can align better to the new world view. This session will also encourage you to look at your immediate and near future testing strategy, as well as your team construct.

UX testing in agile teams | Kimberly Snoyl

UX design takes place prior to the development phase. UX testing, using data, takes place after the product is already built (e.g. using tools like Google Analytics). Kimberly Snoyl will give us her take on UX testing and the possibilities it has during a the development phase. Testing often focuses too much on functionality, instead of ensuring the end user is happy with the product and can use it effectively and efficiently. After this talk you will know what tests you can do, which fit in a sprint, in addition to the functional tests, which will improve the user experience of the product.

How testing is evolving in DevOps | Szilard Szell

Can we be the same testers in a DevOps organization, as in the past? Is that enough if we know test automation, exploratory testing and many aspects of Shift Left; or are there any new aspects, concepts and techniques that a tester should be aware of? Are we able to shape testing with the current competencies, or do we need more? Szliard shares his 20 year journey in testing, process improvement, service design and DevOps, focusing on how testing and quality assurance has evolved, and what the areas a good test professional should be aware of.

Test manager, orchestrator or quality coach? | Gitte Ottosen

Gitte’s session focuses on the changing role of the test manager with the transition to agile. Your work context is under continuous change, so how do you change with it? The role of the test manager is changing – should the role exist any more? For some, the traditional test management role is still 100% relevant but for many of us, the test manager role is transitioning from the traditional test management role to a role focusing on orchestration and quality coaching. This presentation will take you through some of the competences needed to be a good quality orchestrator and quality coach, focusing on both the soft and hard skills which will help you be the best possible support to your team/project/train.

Lost in transformation | Michael Bolton

Every day, more and more organizations are taking on “digital transformation”, leaning on software and machinery to perform jobs, make decisions and solve problems. Machines do work that has traditionally been done by humans, or that couldn’t be done by ordinary humans unaided by machinery. Every day, the reach of what machines can do is extended by technological advancements, growing bodies of data and by human ambition. As software changes, our testing has to respond. Michael Bolton shares what you need to address when going through a digital transformation.

Defects don’t kill you; they make you stronger | Rawad Mansour

We already know that zero defect products don’t exist. Defects that are caught in production can be costly to fix, BUT they are the client’s most concrete feedback on quality. Murex decided to accept the challenge and benefit from this feedback to shape their testing. But how can you benefit from clients’ reported defects to strengthen your test suites? Find out how Murex created a new process to get to the root cause, and take actions on the identified gaps. Plus, the ups and downs of the journey, and how they managed to transform a small team initiative into a global objective.

Have you booked your conference tickets yet? Groups save up to 30% so book your team now and join the whole testing community in Copenhagen.

Thanks to Micro Focus for providing us with this blog post:

The constant surge in digital transformation forces organizations to perform tests on an increasing number of platforms – but still get results as quickly as possible. Manual testing just can’t keep up with the speed of business, so teams turn to test automation. But traditional software testing tools are only effective to a point. Generally, these tools rely on identifying objects on the screen through their internal representation—e.g. coordinates, class name, type, and many other. This method of identifying objects can be very fragile. Even a small change might result in the tool failing to find the object. The drawbacks of these techniques prevent teams from scaling their test automation efforts up to the levels they require.

To that end, the most common test automation challenges include:

• Relentless test maintenance —tests that rely on unique object properties can be susceptible to breaks, thereby making testers perform regular updates to ensure their tests still run on each supported environment.
• Test execution time is too long — even if a test set runs without interruption, it can take a significant amount of time to run all the tests to completion.
• Insufficient test coverage — teams must support an ever-expanding range of platforms, devices, and operating systems, requiring testers to customize the tests for each environment.
• Test creation fatigue — it takes time to build and design effective tests, with much effort required to uniquely identify on-screen objects that are part of the test.

What our research at Micro Focus revealed was that automated object detection with computer vision is key to lowering these barriers.

Computer vision-based AI for automated object detection

Recognizing objects without knowledge of their internal representation was one key objective when we developed our AI engine. This goal was accomplished by combining AI-based algorithms that accurately and consistently recognize objects regardless of device or environment.

For example, a test step might require clicking the shopping cart icon on a mobile app. The AI engine should be able to locate the shopping cart icon on the current screen without knowing:

• If the screen is on a mobile device.
• Whether the device is running Android or iOS.
• If the screen is a desktop browser.
• Whether it’s Chrome, Firefox, Edge, or another browser.

The ability to “Click the shopping cart” step should work under any circumstance with an AI engine. At Micro Focus, we have successfully implemented this in the Micro Focus UFT Family of software test automation products, and the AI techniques used in the engine include computer vision through an artificial neural network and optical character recognition.

Defining computer vision

An AI engine understands a screen’s composition and breaks it down into the unique objects that it contains. Additionally, the AI engine knows nothing about the implementation of the object. It treats the object as an image, regardless of the device or platform it comes from. As such, a powerful CV tool is needed and should be supported by an artificial neural network (ANN), a layered structure of algorithms that classify objects. It will train the ANN with many visual objects, resulting in a model that identifies objects it will likely encounter in applications under test (AUT). Thus, when the AI engine is tasked with locating a specific object, it utilizes the model to identify a match in the AUT

In terms of architecture, we implemented the AI engine as a separate module. Rather than restricting it to a specific product, any of our testing products can theoretically use the engine. Our first product that integrated with the AI engine was Micro Focus UFT One, our flagship test automation tool.

Defining OCR-based identification

Our AI engine also leverages OCR to identify text-based objects. These objects may themselves be part of the test, or they could function as a hint to identify the object’s relative location. This capability is useful if an object appears multiple times on a screen. For example, a login screen might have two text boxes, one for the username and one for the password. OCR helps identify which of the edit boxes is which. OCR can also identify a button by its textual caption.

Maturity of test automation based on computer vision

When we rolled out our new computer vision-based testing capabilities, we worked closely with our early adopter customers. We wanted to learn how they were using the technology and to ensure that clients were receiving the greatest value from it. This collaboration helped us discover new use cases and increase the technology’s reach and coverage.

As with any nascent technology, customers reported multiple edge-case scenarios that helped us improve our implementation and refine our model. Over time, we resolved more scenarios and grew the data set we use to train our model. Our algorithms became more capable and reliable. Due to these improvements, these customers now position AI as the preferred test automation mechanism when possible.

Lowering and removing test automation barriers

AI-based test automation reduces the time it takes to build and design tests because objects are identified simply by looking at them. AI algorithms lower skill barriers because they identify most objects and are hidden from the user. Teams can also use the same test without modification on different devices and platforms. They simply procure an appropriate device and run the test on it as-is. And because the algorithm doesn’t rely on an object’s underlying implementation and properties, the test keeps running even if there is a change. If the test’s flow stays the same, the test will continue to run.

The final barrier yet to be removed completely is test execution time. Tests will always take a finite time to run; hence there is a lower limit on the amount of time they take. However, AI-based testing helps teams test earlier and provides robust mechanisms that parallelize and optimize test execution, reducing the wait time for results.

Micro Focus is a Gold Partner at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now.