Fiona Nic Dhonnacha

Why end-to-end automation testing is crucial for effective DevOps

April 8, 2022 by Fiona Nic Dhonnacha

Thanks to Copado for sharing their insights on automation and DevOps.

One big DevOps misconception is that it is about technology. While a key component, technology alone can’t solve all your software development lifecycle problems. DevOps also requires your business to embrace a new culture. Implementation strategies will differ based on an organization’s size and systems, but most philosophies have a common thread. Almost every expert will agree that end-to-end automation is vital for DevOps. Not only does it let you make the most of DevOps technology, it helps you build a more proactive, transparent DevOps culture.

Automation enhances all components of effective DevOps

If you want to break down the components of effective DevOps, you’ll have to consider both culture and technology. Part of implementing end-to-end automation is looking for how it can add value to each component, supplementing other technologies and helping you build a culture of DevOps.

Culture

Proactivity: DevOps is a proactive strategy as it seeks to reduce problems before they happen. Stakeholders are empowered to come up with resolutions without having to clear them with everyone. Automated testing lets you keep pace with development so you have time to analyze results and fix potential defects early on.

Goal-oriented: DevOps practices aren’t an end unto themselves. They center on an end goal, whether that is improving productivity, reducing bugs, or supporting staff. The developers select the goal and then work backward from it to come up with better strategies, including leveraging automation to eliminate bottlenecks.

Accountability: A culture of accountability should not be mistaken for a culture of blame. With a strong culture, employees will readily communicate problems rather than cover them. Automating tests and validation gives employees more room to explore and innovate, free from the fear one small mistake will bring the whole project crashing down.

Communication: Stakeholder alignment is critical in DevOps approaches. Everyone involved needs to understand the goals of the project, what is required of them, and how they can further the mission of the company. With automation comes reporting, which can improve transparency and facilitate communication between teams.

Technology

CI/CD: Continuous integration and continuous delivery pipelines are crucial for DevOps as they ensure the timely delivery of updates and improvements. It also allows for easier detection of errors due to how often code repositories are checked and tested.

Continuous testing: Continuous testing goes right along with CI/CD and automation. A test accompanies every line of code so you can ensure that it performs as expected. With continuous testing, you locate issues earlier and prevent widespread problems.

Version control: You must track the changes made in your code and ensure everyone follows the same standards. Version control makes these changes transparent and easier to understand. When there is a problem, you can locate and isolate it.

Automation tools: Manual steps are not scalable – and that’s vital in a DevOps environment. If a process is high-volume, repeatable, and predictable, you want to find a way to automate it so your developers can focus on higher-value tasks. You can use automation to support your CI/CD, continuous testing, and version control systems at scale

DevOps will mean different things to different organizations. However, all the above components should be included, regardless of industry type or software needs. With the above, you set the foundation for end-to-end automation and position your organization to make the most of its benefits.

Where end-to-end automation testing fits in

End-to-end automation has applications throughout the development lifecycle, but one of its key uses is to speed up testing. Testing is a necessary but time-consuming part of any DevOps cycle. The size of modern systems and software make it impossible to adequately scale up manual testing.

Robotic process automation is an ideal tool for this. RPA-based testing can be integrated into the CI/CD pipeline. As you deliver changes, you’re also providing the tests that verify their efficacy. Testing is one of the most common bottlenecks in the CI/CD pipeline, but RPA helps overcome these issues proactively. Of course, when you’re considering these tools, it’s important to choose options that are:

System agnostic: You should not have to use a different test for every platform, app, or cloud you work in. Any end-to-end automation testing tool must be system agnostic to support the diversity of your infrastructure.
Low- or no-code: Development is no longer an isolated part of an organization. Our world runs on software. The people who handle your big picture goals and business needs should create solutions within the software that works for them. However, they should also be following strong DevOps principles. A low code or no code testing solution helps them make sure that their programs run without bringing in an expert.
Self-healing: Your software will change over time. That means your tests will need to change too. A self-healing test automation tool will notice these changes and adapt your tests to ensure they continue to work through updates. This component makes these programs truly scalable to keep up with rapid system growth.
SaaS-based: A software as a service solution is best for deploying end-to-end automation. It comes with its own infrastructure, so you don’t have to worry about a long ramp-up to get your program up and running.

End-to-end automation will support your DevOps principles with continuous testing in a CI/CD pipeline and beyond. Tests are a crucial but time-consuming part of any software development initiative. By leveraging a program that runs on RPA, you can reinforce the scalability and flexibility that DevOps philosophies demand.

Copado is a Gold Sponsor at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by April 22nd and save 10% on individual tickets; up to 35% on group bundles. See you in Copenhagen.

15 things you’ll learn from EuroSTAR speakers

April 5, 2022 by Fiona Nic Dhonnacha

It’s our first in-person event in 2 years, and we’re so excited to bring the entire testing community together again! This year the EuroSTAR conference is bigger and better than ever before. We’ve got a programme packed with inspiring keynotes, interactive tutorials, and buzzing track talks – brought to you from 70 world class speakers.

You’ll benefit from soaking up all that incredible knowledge, and gain practical takeaways, new ideas, and fresh perspectives that you can bring home with you after the conference. We’ve put together just a small taste of what you can expect this year. Get your EuroSTAR ticket now, and start planning your conference diary.

1. Understand that testing is never finished

It is hard to understand one thing in isolation: things need to be connected to each other in order to generate a good understanding. This means your understanding of testing is never finished, as you learn more about the product and its relations – and the relations is key. Get a broader understanding and see examples of learning paths in a complex world at Rikard Edgren’s keynote. Get comfortable with the feeling that testing is difficult, and the fact that it should be. You’ll also gain a deeper understanding of your software, your situation – and all the relations.

2. Get ready to meet the needs of the next economy

Smita Mishra shares what she’s learnt about the next economy, and what technology changes we are seeing across users. How does this impact what testers do? Do we understand the users of these new technologies? Are we ready to meet the needs of our customers of the next technology? Get answers to these questions, and a whole lot more. Let’s learn what the world thinks could be technology risks, and how testers can align better to the new world view. This session will also encourage you to look at your immediate and near future testing strategy, as well as your team construct.

3. Learn how to go through a digital transformation

Every day, more and more organizations are taking on “digital transformation”, leaning on software and machinery to perform jobs, make decisions and solve problems. Machines do work that has traditionally been done by humans, or that couldn’t be done by ordinary humans unaided by machinery. Every day, the reach of what machines can do is extended by technological advancements, growing bodies of data and by human ambition. As software changes, our testing has to respond. Michael Bolton shares what you need to address when going through a digital transformation.

4. Scale your team like a pro

As your team scales, the nature of challenges will change, Just like a bonsai tree, you can’t leave a team’s growth and shape to chance without a plan. One of the major components that a bonsai represents is that of balance. How can you achieve the right balance in a remote, multicultural team, and achieve your purpose ? Seema Prabhu shares her experience with her team changing shape as they scaled, the lessons learnt, and tools she wished she had during this phase of growth.

5. Learn how to respond to external changes

The world is constantly changing, and everything is impermanent. Most especially after the last two years, we have really been forced to come to terms with how quickly and drastically things can change. How will external changes shape our teams and our work? How can we shape ourselves proactively in order to respond to changes, make changes of our own, and even thrive? Alex Schladebeck looks at what factors are at work now, and what known and unknowns we have. You’ll also learn what kinds of effects they will have on how we work, and the roles of testers and software professionals.

GET TICKETS

6. Sell and shift your testing at work

As testers who have an exploratory approach to testing, it can be challenging to gain acceptance and buy-in from leadership. Often, people you are trying to sell to are left asking, “What the hell kind of testing is that?”… and not in a good way. Nancy shares her experience with exploratory testing, and shifting testing to new methods. This high energy session will send you back to the office with the tools you need to help sell and shift your testing at work.

7. Develop your critical thinking skills

We expend huge efforts in training people in various testing skills. Yet, we often fail to train ourselves in the most important testing skill of all – how to think critically about what we are testing. When we test, we are vulnerable to cognitive biases and thinking traps that can catch out even the most seasoned tester. In this workshop, Andrew Brown shares the training course he’s developed, to show you how to improve your testing by enhancing your critical thinking skills, and addressing biases and thinking traps.

8. Set up and write your own automatic accessibility tests

Accessibility testing tools are of great help in making an accessible website, and shaping a more inclusive web. In this workshop you’ll learn how to set up and write your own automatic accessibility tests using Axe and Cypress, and cover some of the most common accessibility errors that can be discovered automatically – and how to fix them. This workshop will be presented ensemble style – mob testing with small groups where the whole group works on the same thing, at the same time, in the same space, and at the same computer.

9. Use defects to your benefit

We already know that zero defect products don’t exist. Defects that are caught in production can be costly to fix, BUT they are the client’s most concrete feedback on quality. Murex decided to accept the challenge and benefit from this feedback to shape their testing. But how can you benefit from clients’ reported defects to strengthen your test suites? Find out how Murex created a new process to get to the root cause, and take actions on the identified gaps. Plus, the ups and downs of the journey, and how they managed to transform a small team initiative into a global objective.

10. Navigate the new normal

Learnings from the past 1.5 years requires us to answer a fundamental question: how do we shape a post pandemic testing career? After all, there have been seismic shifts in customer behavior, technology solutions, business models and ways of working. It’s changed everything, especially testing. Ryan Volker will help you navigate the new normal, and get you better equipped to design your post pandemic career. Learn what changes will stick around – and how you can profit from it, plus how new work and old work will shape your future testing.

11. See why failure is a good thing

Fail. Red Build. Outage. Regression… if you’ve ever worked in DevOps, these phrases probably fill you with dread… but they shouldn’t! Failure is vilified, it’s judged, sometimes it’s punished… And it’s the default. Success is the exception. Dylan Lacey is here to show you that failure is a good thing, and how confidence without failure is false confidence. He’ll present concrete examples of how failure helps organisations be more confident. Learn how to have more failures, failure anti-patterns, and pernicious failure psychology.

get tickets

12. Go from survive to thrive

A new job, in a new industry, a new city, a new tech stack, and no PC… sound overwhelming? Chris Armstrong started a new job just like this – and with no PC he had no email or access to training/onboarding/tests/codebase. He wasn’t even introduced to the team. He had a copy of industry-standard documentation, and his boss went on leave. What would you do? Chris shares how he turned this situation around, and how he now ensures that new starters he works with don’t ever have to go through what he did. This is how you go from surviving to thriving.

13. Learn about the 8 testing senses

Tools become extensions of our perceptions, our cognition, and our physical movements. Good tools enhance our flow as we work. Poor tools impede our flow, and even become blockers. In understanding interactions between people, their tools, and their real and virtual worlds, Isabel Evans asks: what if we engage with not just five but all our eight senses? Learn how tools tell us what to do, how tools become an extension of our bodies, and how tools use or ignore our senses. This affects what we think about when testing products, and when choosing our own tools for supporting testing.

14. Automate BDD scenarios with SpecFlow

This workshop helps you to speed up with automating BDD scenarios with SpecFlow, the official Cucumber implementation for .NET. Gáspár Nagy will start with a very brief introduction to BDD/ATDD, and what are the most important characteristics of good BDD scenarios, before jumping into coding in order to learn about the most important features of SpecFlow. You’ll get plenty of hands-on exercises where you can practice how to automate and execute BDD scenarios with SpecFlow, and see how the test-first approach can help you to get quick feedback about the quality.

15. Get into a T-shape

Software development is in continuous change, and so is testing. As a tester our role has evolved into multiple forms, mostly focusing on being more T-shaped. But how do you become more T-shaped, and what is T-shaping? Gerard talks through the various forms of T-shaping within DevOps engineering, like CI/CD enabling, software development, and operations, and show you how you can become more T-shaped – by learning that achieving knowledge is about setting goals

Whew. That’s just a tiny taster of what’s on offer at EuroSTAR this year. This is a space where you can ask questions, share your ideas, and get help from the testing community to solve your problems. You’ll walk away with the knowledge you need to take your testing to the next level.

Book your ticket now and get 10% off with our Early Bird offer – this ends April 22nd, so book now.

How to automate functional UI testing using the I-A-C pattern

March 28, 2022 by Fiona Nic Dhonnacha

This article was provided by Alfonso Nocella, Co-founder and Sr. Software Engineer at Maveryx.

Automated Functional UI Testing

We generally used to consider Functional UI Testing the most critical type of testing because, first of all, an application must do WHAT it is supposed to do. And indeed, it verifies that an application works as expected through its user interface.

Or, if you prefer, according to Glenford Myers, Functional UI testing means to verify the software through its UI with the intent of finding errors (i.e., “find discrepancies” between the program and its specification).

Given these points, Functional UI testing verifies, from the end user’s perspective (through the user interface), that the software behaves according to its functional requirements specification without knowing the internal logic and implementation (black-box). In simple words, Functional UI testing verifies an application in the same way an end-user would.

Also, Functional UI tests are effective because they go “end-to-end” from the UI layer. Indeed, end-to-end means testing all the different parts of an application from the beginning (the user interface) to end (the underlying services, the database connectivity, etc.).

From the automation point of view, Functional UI tests imitate users’ actions such as clicking on a control, entering text, scrolling a page, and so on by also comparing the actual outputs with the expected outcomes.
With this in mind, in automated Functional UI scripts, working with user interface elements, a tester should always use this pattern:

IDENTIFY: locating the elements to use
ACT: performing the requested (test) actions
CHECK: comparing expected results vs. actual results

In fact, this pattern recurs continuously in an automated test script.
This paper will explain how to write automated Functional UI tests using this pattern.

The I-A-C pattern

In an automated Functional UI script, each test “step” consists of two parts: “action” and “checkpoint.” An “action” is an interaction between the test tool and the UI object-to-test. An example of action is clicking a button or entering a text into a text field.
Besides, an action consists of a “target”, a UI object-to-test, and a “command”, a test action to perform (e.g., “click”, “type”, …). A command may have inputs (e.g., type “some text”) or not (e.g., “click”).

A “checkpoint” compares the actual outcome from the application under test and the expected one. For example, the predicted result could be that typed text is in the text field.

Hence, writing an automated Functional UI script, a tester shall:

(uniquely) IDENTIFY the UI object-to-test
set the test ACTion to perform
CHECK that expected results and actual results match
All this is what we call the I-A-C pattern.

For this article, we will use the login system of the OrangeHRM demo software. (See Figure 1).

Learn how to automate functional UI testing using the I-A-C pattern, and create more effective automated tests. — Figure 1 – OrangeHRM login sample

Test case design

First of all, let’s start designing the test case.
On a login system, a tester wants to insert valid credentials (username and password), and after clicking on the login button, she automatically wants to log in.

Schematically:

Enter a valid username
Type a valid password
Click the login button
Check user is successfully logged in

Let’s automate this test case by using the I-A-C pattern.

IDENTIFY the UI object-to-tests

The first step is to locate the test objects.
So, in the OrangeHRM login test case (Figure 2)

username text field
password text field
login button

Selenium and Selenium-based tools use “Locators.” Basically, a locator enables testers to identify the web element to interact with.
Selenium offers several different methods like by className, cssSelector, id, name, XPath, etc.

WebElement usrName = driver.findElement(By.id(“txtUsername”));
WebElement password = driver.findElement(By.name(“txtPassword”));
WebElement login = driver.findElement(By.xpath(“/html/body/div[1]/div/div[3]/div[2]/div[2]/form/div[5]/input”));

Other tools use “pre-recorded” GUI Maps (or Object Repositories) to locate objects.
A GUI Map represents an abstraction of the structure of the user interface. Hence, it contains the logical names and physical descriptions of the UI objects to test.

For example, a tool based on this technology reads an object’s description in the pre-recorded GUI map (Figure 3) and then looks for an object with the same properties in the tested application.

Other tools like Maveryx do not use any locator or map: the UI elements to test can be described into the test scripts as they simply appear in the application (on-screen).

In this example, the username text field is identified by its placeholder “Username” (Figure 4).

//the Username text field
GuiText usrName = new GuiText(“Username”);

Also, the password text field can be identified by its placeholder “Password” (Figure 5).

//the Password text field
GuiPasswordText password = new GuiPasswordText(“Password”);

The same applies to the login button, which can be identified by its caption “Login” (Figure 6):

//click Login button

GuiButton login = new GuiButton(“LOGIN”);

So, the test objects defined in the test script are identified directly at runtime without using any pre-recorded GUI Map.

ACT on the UI object-to-tests

Once the test objects are located, a tester wants to perform the relevant test ACTions on them, such as clicking on a widget, typing some text, selecting an item in a drop-down list, etc.
In our example, we would enter some text into the username and password fields and click on the login button.

For instance, to execute these actions with Maveryx, a tester can use setText() and click() methods:

//the username
String username = “Admin”;

//the Username text field
GuiText usrName = new GuiText(“Username”);

//set the username
usrName.setText(username);

//click Login button
GuiButton login = new GuiButton(“LOGIN”);
login.click();

Obviously, the same goes for Selenium or other tools.

CHECK the results

Now that the test interacts with a GUI object, it’s time to verify that the expected output is produced.

Assertions are the best tools to verify that the expected test results match the actual results. They are assumptions on the application under test that must always be true. If any assertion fails, the test will fail. Similarly, if all assertions pass, the test will pass.
Hence, assertions help testers to quickly check if the application under test behaves as expected or not.
For example, if you write a method that calculates the sum of two numbers (e.g., 3 + 5), you might assert that the sum is correct (= 8).

Each assertion contains a boolean expression that you believe will be true when executed. The experience proved that assertions are one of the quickest and most effective ways to detect and correct bugs.

There are several assertions libraries like JUnit, Hamcrest, AssertJ, etc.
For instance, JUnit provides assertEquals() and assertNotEquals() methods to compare the equality and inequality of values. So, the assertion passes when values are equal; otherwise, it fails and throws an AssertionError.

The first parameter passed to the assertEquals method corresponds to the expected value, and the second parameter is the actual value.
In functional UI testing checkpoints include:

values checking: a string is entered, a message is displayed, a drop-down contains a list of items, a web element has a given property, etc.
property checking: a UI object is enabled, is editable, focused, etc.
presence checking: a UI object is present or not in the current user interface

Our example will check if the username and password text field contain the entered search key.

//the username
String username = “Admin”;
//check that the username has been correctly inserted
assertEquals(username /* expected /, usrName.getText() / actual */);

//the password
String pwd = “admin123”;
//check that the password has been correctly inserted
assertEquals(pwd /* expected /, password.getText() / actual */);

If we would check that the “login” button is enabled, we can use: assertTrue(login.isEnabled() /* actual status */);

We could also check the URL of the landing page after logging in
//the Dashboard page URL
String dashboardURL = “https://opensource-demo.orangehrmlive.com/index.php/dashboard”;
//check the Dashboard page URL
assertEquals(dashboardURL, new GuiBrowser().getCurrentPageUrl());

Another way to check that an application behaves correctly is to verify that a UI object is present or not.
We can also verify that the “Welcome” message popped up to check the successful login (Figure 7). For this purpose, testers can use the waitForObject() method:

//check that the “Welcome Shinchan” is present
new GuiHtmlElement(“Welcome Shinchan”).waitForObject(5, 1);

The waitForObject() function waits until the given object exists. It returns if successful or raises a (catchable) ObjectNotFoundException exception on failure, i.e., if the function times out and the UI object is not displayed. By this approach, you can use this method to check if an object exists (i.e., is shown) or not.

Conclusion

Functional UI testing is essential because it verifies that an application does what it is supposed to do through its user interface, as the end-user would. Also, it allows testing an application end-to-end from the start (the UI layer) to end (services, DB, etc.).
From the automation point of view, Functional UI scripts consist of two main blocks: UI interactions and assertions.
UI interactions consist of firstly identifying the UI objects to test and then acting on them.
On the other hand, assertions allow comparing expected results from specifications to actual results from the application-under-test.

A tester should keep in mind the I-A-C pattern to write good automated Functional UI tests:

IDENTIFY the UI object-to-test;
ACTing on it;
CHECK that expected outcomes and actual outcomes match.

In conclusion, this pattern allows creating more effective automated tests, ensuring maximum test flow control and at least more readable and maintainable scripts.

Maveryx is exhibiting at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by April 22nd and save 10% on individual tickets; up to 35% on group bundles. See you in Copenhagen.

Author

Alfonso Nocella

Co-founder and Sr. Software Engineer at Maveryx, Alfonso led the design and development of some core components of the Maveryx automated testing tool. He collaborated in some astrophysics IT research projects with the University of Napoli Federico II and the Italian national astrophysics research institute (INAF). Over the decades, Alfonso worked on many industrial and research projects in different business fields and partnerships. Also, he was a speaker at several conferences and universities.

Today, Alfonso supports critical QA projects of some Maveryx customers in the defense and public health fields. Besides, he is a test automation trainer, and he takes care of the communication and the technical marketing of Maveryx.

10 EuroSTAR tutorials to elevate your testing

March 24, 2022 by Fiona Nic Dhonnacha

This year’s EuroSTAR tutorials feature 10 in-depth sessions from expert speakers. This is your chance to learn new skills through interactive group work, practical examples, and hands-on exercises. Our tutorial trainers will share their triumphs – and mistakes – so that you learn from real experience.

Want to learn more about how biases affect your testing? Improve your critical thinking skills? Learn new concepts and testing tools? This is where it all happens.

Full-day tutorials

1. Shaping testing – A simulation in scrum | Fran O’Hara

It’s reported that approximately 80% of agile organisations are using Scrum and its hybrids. However, the majority of organisations are still at the earlier stages in their evolution. One of the challenges they face is quality and testing. In this simulation at Fran’s tutorial, you’ll work as part of a Scrum team focusing solely on the test and quality-related aspects.

You’ll get insights, knowledge, and skills to help shape your team’s approach to quality and testing throughout the agile lifecycle. You’ll also be faced with a series of challenges/scenarios often faced by testers during the sprint. You’ll walk away with a toolbox of ways to solve the typical problems that impact on test and quality in Scrum.

book tickets

2. Developing critical thinking skills for testers | Andrew Brown

Critical thinking skills are essential to testing – but rarely taught. We’re also vulnerable to cognitive biases and thinking traps when we test – that can catch out even the most seasoned tester. One of the most common thinking traps we face is that we tend to think in ways that confirm our existing beliefs, rather than challenging those beliefs.

This tutorial will reveal your vulnerability to this bias, then show you how to mitigate it. Throughout you’ll learn exercises to explore your biases, and how reasoning does not use logic – or at least not rational logic. Rather, we use logic based upon social permissions and cheater detection. You will use this to explore how to write better tests by rephrasing testing requirements in terms of social logic. At the end, you’ll have ways to improve your testing by enhancing your critical thinking skills.

view tutorial

3. Test design with data combination testing and classification trees | Rik Marselis

Test case design is one of the core competences of the quality engineering & testing profession. If you want to properly shape your testing, which test design technique(s) do you use? And is it effective and efficient? What coverage can you achieve? Rik explores the Data Combination Test design technique, which uses Classification Trees.

This technique can be combined with 3 different coverage types, so you have a great way to align with the risk level of the application. Join Rik’s tutorial on DCT & CT and start applying his techniques, the very next day you return to work.

see tutorial

A group of attendees at EuroSTAR Conference working together at Rik Marselis's tutorial

4. Automate BDD scenarios with SpecFlow | Gáspar Nagy

SpecFlow is the official Cucumber implementation for .NET, and if you want to speed up automating BDD scenarios with SpecFlow, this workshop is for you. Gáspar will give you a very brief introduction to BDD/ATDD, the most important characteristics of good BDD scenarios, before getting into coding in order to learn about the most important features of SpecFlow.

This is your opportunity to learn about the BDD automation workflow, and see how the test-first approach can help you to get quick feedback about the quality. You’ll also see on what levels you can automate the application with BDD, and how to make a good mix of them to get a sustainable testing strategy.

get tickets

Half-day tutorials

5. Questions, questions | James Lyndsay

At this interactive workshop, you’ll learn techniques to obtain more useful information quickly, ask the right questions at the right time, and learn where to go next. James will show you how to recognise types of questions, the contexts in which you might use them, and the people you might ask.

The tutorial also looks at ways you can use questions poorly: loaded and leading questions, questions designed to trick or trip, questions which go over the same ground, and other anti-patterns. You’ll know how to ask more purposeful and focused questions, and see the patterns, gaps and opportunities in your own questions.

view tutorial

6. From business workflows to automated tests | Anne Kramer

With the deployment of agile development practices, QA & testing teams are challenged by the acceleration of production releases, and the imperative of test automation. These challenges make test relevance, and the alignment of these tests with business needs, even more crucial. After all, what would be the economic justification of investing in automation and maintenance of tests that do not properly reflect the business needs?

This tutorial introduces you a visual TDD approach (Acceptance Test Driven Development). In this approach, the test requirements are expressed in graphical workflows. You’ll build a graphical test workflow for a simple functional scope proposed as a practical exercise. Learn how to express business needs and test requirements in the form of graphical workflows and appreciate the collaborative contribution of this visual representation.

see tickets

Speaker teaching a tutorial class at the 2019 EuroSTAR conference

7. Exploring context driven testing & exploratory testing | Nancy Kelln

More testers are gravitating towards using exploratory testing and context driven testing techniques in their organizations. However, as testers start to embrace these testing methodologies, they are uncovering questions in their implementation. In this half-day workshop, you’ll explore the various aspects of testing including test planning, test design, test execution, and test reporting, from the exploratory testing mindset.

You’ll also learn how to prepare your organization for the shift from more traditional methods to exploratory testing methods. You’ll leave with an understanding of how to implement exploratory testing concepts through all the phases of test planning, design, execution, and reporting.

view tutorial

8. Automatic accessibility testing for all | Cecilie Haugstvedt

In this workshop you’ll learn how to set up and write your own automatic accessibility tests using Axe and Cypress. You’ll write both unit tests and integration tests, and will look at what to test at each level. Cecilie will also cover some of the most common accessibility errors that can be discovered automatically, and learn how to fix them.

This workshop will be done in ensemble style – mob testing with small groups where the whole group works on the same thing, at the same time, in the same space, and at the same computer. You’ll leave understanding how to implement exploratory testing concepts through all the phases of test planning, design, execution and reporting.

get tickets

9. Simplicity: distilling and refining test communication | Rick Tracy

This workshop, which uses the idea of simplifying how we communicate among ourselves and others, aims to distil the core results of testing into a much easier to understand message. Often we end up using too much industry jargon and test language, and this ends up diluting our message and the value of our communication. Once we’ve distilled the message to someone direct and valuable to all, we can much better target specific audiences with a refined message.

You’ll leave the workshop with a good understanding of why your valuable messages don’t always come across, what you can do about it, and how you can apply these skills regularly to your everyday work. You’ll explore several techniques to make a clear overall message, and to make refined valuable messages, no matter who the stakeholder is.

see tutorial

10. The good, the bad, and the biased | Emma Lilliestam & Hanna Schlander

How does the brain work? Why do biases exist, and what are the pros and cons of biases? In the workshop you’ll learn about two thinking systems in the brain, and four categories of biases. We are all biased. Biases affect our everyday lives, both at work and at home. By learning more about them we can expand our view, shape our minds, and improve our testing abilities!

You’ll get the chance to deep dive into biases in groups, and discuss a selection of our favorite biases, before presenting your findings to the group.

view tickets

Excited to start learning? It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Soak up knowledge from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by April 22nd and save 10% on individual tickets; up to 35% on group bundles.

Remote ERP testing how-to

March 14, 2022 by Fiona Nic Dhonnacha

We are delighted to share this article from Panaya, Gold Sponsors at EuroSTAR 2022.

3 best practices based on successful, real-life projects

In light of the new business reality, organizations must continue to deliver their products and services with the highest quality without any setbacks. ERP reliability is a significant component of overall enterprise performance, and ERP Testing is the key to having just that. But a successful testing project requires a set of tools and methodologies that synchronize in perfect harmony.

Successful ERP testing projects also require business and IT organizations to be completely aligned. Challenges increase when employees across the globe need to collaborate with their peers remotely as if they were across the hall.

Staying focused and ready to deliver in uncertain times

By now, we all know the challenges of navigating an ongoing, global emergency. Priorities are shifting quickly, and directives become chaotic, while IT and business users are scrambling to catch up. How do you stay focused on a cross-organizational testing project to meet deadlines when working remotely?

What we learned from our test dynamix customers working remotely

Panaya’s ERP testing experts have come up with some best practices for executing a globally dispersed testing project. They are based on real-life success stories of multinational corporations. Here are the top 3 principles of remote testing efficiency and the projects that successfully implemented them.

1. Enabling effective tester collaboration

A critical element to successful testing, is effective communication between all stakeholders. Testers have to know what to test, when to test, how, and with which data.

Developers have to know quickly that a defect is assigned to them; they need to effortlessly understand the root cause and steps to reproduce it, engage testers in discussion, etc.

Test managers need to share progress, and receive feedback from testers. They must be able to schedule reminders, communicate with project team members, and do more to get their project moving forward.

And while standard communication tools like Zoom and Skype are easy to use, they are not testing tools. Collaborative testing tools could provide you with automated handover between testers across the same business process such as:

“Ready to Test” when it’s a user’s turn to test within the business process,
“Retest” notification to testers upon defect resolution, and
“Close” enforcement to testers upon retest success.

One more aspect of efficient testing is to have easy to use test evidence which can help you deliver granular information and save an enormous amount of time while improving collaboration between testers and between testers and developers. Having a collaborative tool enabling Self-generated test evidence at the click of a button reduces the back and forth between testers and provides a record of steps to reproduce opened defects.

Panaya Test Dynamix (TDx) supports collaborative communication between QA, developers, and business users for joint reviews and the detection of errors early in the testing cycle.

We loved Panaya TDx for its collaboration features. It is a user friendly, cloud based solution that offers easily repeatable test scenarios between similar projects.” We would recommend it to any other organization running SAP”
pam brown, erp business process organisation at bruker

Read the full Bruker success story.

2. Adopting a holistic E2E view of the entire test cycle

Test managers should always be able to see the full picture and take a holistic approach to navigating their ship through a sea of obstacles. When testing remotely, this is often easier said than done. But with the right tools in place, visibility and control can be effortless.

For example, having a holistic defect management approach can significantly reduce global test efforts. Being able to identify defects early in the test cycle and quickly relate them to any future or on-going tests, could save valuable time running tests that are bound to fail. Having a central defect management system can accelerate the entire execution process. Features like System-Wide Defect Management block compromised tests and notify testers on potential issues. This, in turn, minimizes idle time, improves testing efficiency, and removes a recurrent bottleneck of testing.

“Our key business users are focused on their daily activities. They are not technical testers. We trained them once on using Panaya and now there is less effort put into email communications around testing requests. The project manager schedules and monitors the workflow on a dashboard and the system notifies key users when it’s their turn to test between similar projects. We would recommend it to any other organization running SAP.”
Phuong vo vinh, sap applications project manager at efs

Read the full Etablissement Français du Sang (EFS) success story.

3. Choosing the right tool

Too many companies are focusing more on the technical side of testing instead of on business process testing. This, in turn, creates a disconnect in the organization between IT and business users.

Specifically, for ERP testing, the focus has to shift towards business process testing and testing tools designed with non-technical users in mind.

In essence, this is not only critical for the success of your testing project, but it is also a great way to accelerate it. Equipping your teams with the right tools can make remote testing easy and effective. Choose a tool designed for business process testing, focus on a dedicated SaaS solution that is installation-free, easy to onboard and use, and business-user friendly, like Panaya Test Dynamix (TDx), and you’re on a sure path to success.

“We are using Panaya as a Testing Factory, so we are providing testing as a service to all our teams. This has proven to be especially effective with our offshore teams. The collaboration capabilities of Panaya helped us achieve our goals. We are now testing everything centrally – both on-site and offshore – with automated processes and test evidence. This ensures that our teams get very reliable feedback and can quickly and safely resolve and release.”
Sebastien giraud, delivery & test manager at lapeyre.

Read the full Groupe Lapeyre S.A success story.

Panaya Test Dynamix (TDx) customers have been running large-scale UAT projects remotely for years. They are able to use our technology to cut project cost and duration by 30% to 50%. They do it without traveling to the same location, or meeting for tens of hours in virtual rooms.

Are you currently testing remotely or have an upcoming remote testing project? ERP testing with Panaya Test Dynamix is now free for 14 days. Onboarding is quick and easy. Key features include business processes testing with automatic handover between users, as well as an intuitive real-time dashboard with actionable insights and automated defect management tools. Sign up now.

Panaya is a Gold Sponsor at EuroSTAR 2022. It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by April 22nd and save 10% on individual tickets; up to 35% on group bundles. See you in Copenhagen.

EuroSTAR 2022 Headliners

February 21, 2022 by Fiona Nic Dhonnacha

Our EuroSTAR 2022 headliners include keynote speakers who are world renowned testing experts. They’ve got the experience and insights that you’re dying to learn from, in automation, agile, AI, DevOps, security, test strategy and much more. Introducing Rikard Edgren, Smita Mishra, Michael Bolton, Alex Schladebeck, and Nancy Kelln.

5 incredible speakers, 5 energy-fueled talks to get you excited about what’s new in software testing. They are bringing ideas, answers, thoughts, experience – and maybe a few questions to answer together. Get your ticket now and join us in Copenhagen.

My Essence of Testing: Understanding Relations

Rikard Edgren

Many testers work in a situation where quality is important, but there’s not enough time to make all the tests you would like. In his keynote, Rikard shares his 21 years of experience in testing. He explores what makes it difficult, and how, with experience he finds more important information faster. Learn how to change your mind more easily when the current methods are no longer the best.

It’s all down to the understanding of software, of testing, of the domain, and the concrete situation you’re working in. Your understanding of testing is never finished as you learn more about the product and it’s relations – and the relations is key. You can never understand one thing in isolation. Get a broader understanding and see examples of learning paths in a complex world.

Key takeaways:

Get a broader understanding of understanding
See examples of learning paths in a complex world
Reflect on the relations that matter to you

see talk

Testing Software for the “Next” Economy

Smita Mishra

Today, we have hi-tech advances happening where exclusive software is being built to support fancy projects like reaching and exploring mars. We also have mindful technologies evolving to serve sustainable development use cases in the area of climate tech, agri tech, edtech, etc, to make the world a safer place. There are businesses preparing to serve the next billion users who will move out of poverty and have economic access to spend on technology for the first time

How does this impact what testers do? Smita shares what she’s learnt about the next economy, and what technology changes we are seeing across users. Do we understand the users of these new technologies? Are we ready to meet the needs of our customers of the next technology? Learn the potential technology risks, and how we as testers can align better to the new world view.

Key takeaways:

How is the technology going to shape up for the diverse set of users?
Key quality factors that the new age software are looking for
Redesigning the test strategy (including team and tools) for the “next” age software

see programme

Lost in Transformation

Michael Bolton

If we intend to be responsible problem-solvers, we should ask if our solutions have really addressed the problems. Are the problems left unsolved? Or have we perhaps introduced new ones? Are there things we’re forgetting in the rush? As software changes, accelerating and extending our capacity to do things in the world, we need testing and our ideas about it to respond to those changes. As we go through our digital transformations, Michael asks: are there things we’re forgetting in testing too?

Key takeaways:

If we intend to be responsible problem-solvers, we should ask if our solutions have really addressed the problems.
As software changes, accelerating and extending our capacity to do things in the world, we need testing and our ideas about it to respond to those changes.

get tickets

Drawing Lines Around the Fog: The Shape of Testers, Teams and the World

Alex Schladebeck

The world is constantly changing, and everything is impermanent. After the last two years, we have been forced to come to terms with how quickly and drastically things can change. As IT professionals, we are aware of the intrinsic changeability of projects, contexts and our business, but the events of the last couple of years have put this into sharper focus. How will external changes shape our teams and our work?

How can we shape ourselves proactively in order to be able to respond to changes, make changes or our own and even thrive? Alex looks at what factors are at work now, and what kinds of effects will they have on how we work, and the roles of testers and software professionals. She will also look at concrete activities on an individual and company level, to best prepare ourselves for a nebulous future.

Key takeaways:

How will external changes shape our teams and our work?
How can we shape ourselves proactively in order to be able to respond to changes, make changes of our own and even thrive?

see session

What the Hell Kind of Testing Is That?

Nancy Kelln

Many organizations are not ready to accept the differences between exploratory testing and more traditional testing methods. As testers who have an exploratory approach to testing, it can be challenging to gain acceptance and buy-in from leadership. Often, people you are trying to sell to are left asking “What the hell kind of testing is that?” and not in a good way.

Nancy Kelln has implemented exploratory testing concepts at various organizations over the past ten years – Her experience spans implementing these concepts as a tester, a test lead, and as a manager. She also has experience in selling exploratory testing to testing teams, management, and stakeholders. Nancy examine the lessons learned when shifting testing to new methods, what worked, what didn’t, as well as how to recover when things go awry.

Key takeaways:

Tricks to pivot your testing to implement new ideas in your organization
How to recover when newly implemented ideas go sideways
How to reignite your spark and passion for great software testing.

view programme

If you book your ticket now, you’ll save 20% on individual tickets; up to 43% on group bundles. This offer ends Feb. 28th so grab your tickets now and join us in June for a massive celebration of testing!

Unleashing the next level of quality assurance

February 17, 2022 by Fiona Nic Dhonnacha

We are delighted to share this article written by Khimanand Upreti and Hannes Färberböck of Nagarro, Gold Sponsors at EuroSTAR 2022.

According to a Harvard Business Review survey among 250 business leaders, 66% of respondents felt that their company’s future depends on software quality. As per Nagarro’s internal surveys, 78% of enterprises strongly believe they need more efficient QA organizations to respond to disruption.

Quite evidently, Quality Assurance (QA) is an essential part of the enterprise software lifecycle. But what does the future hold for QA? Today, QA needs significant development to meet the evolving requirements in the following aspects:

1. Hyper-automation

Gartner has predicted AI engineering technologies and hyper-automation to be the major technology trends for 2022. Many organizations are still cocooned and traditional in their automation approach. We must explore how best we can use hyper-automation for testing.

2. Design thinking

Most QA organizations or teams lack “design thinking” as they typically follow a functionality-centric approach instead of a user-centric one. It is not uncommon for even products with good functionality to not manage decent app store ratings. What can we do to develop a user-centric approach to quality engineering?

3. Lean QA

There is an absence of lean QA. Too much time is spent in the testing phase of the development lifecycle without enough tangible or measurable impact. Testing and development are still highly decoupled, leading to a lot of avoidable practices such as handover and unnecessary alignments.

4. Proper, process-based knowledge transfer

In many enterprises, QA is people-dependent, with the critical know-how or intelligence about any aspect remaining concentrated among a select few individuals. The organization can lose crucial information in the process as such knowledge is often shared at an interpersonal level than at a process level. Knowledge transfer is a key area that must be improved as individuals leave or take on other roles.

5. Continuous improvement

Perhaps the biggest hurdle in modern QA teams is the lack of continuous improvement. The secret behind any world-class software is a continuously monitored and adaptive quality assurance system backed by robust standard operating procedures. Yes, most enterprises typically have their SOPs, but there is always room for improvement, especially ensuring that the recommended practices are adopted and monitored correctly.

Given these challenges, how can QA grow its prominence in today’s rapidly changing software ecosystem? Let us find out!

Testing the uncertainty: How will AI and autonomous systems impact QA?

Gartner predicts that more than 50% of organizations will compete using advanced analytics algorithms globally, with many of them being AI-based.

Traditional QA methods and approaches are aligned with – surprise, surprise – traditional applications, which are deterministic logic-driven. This means that for each determined input, there is a determined output. So, one can predict the output for any given input.

On the other hand, AI-based models are probabilistic logic-driven. This means that for a given input, the output is unpredictable. The output of an AI-based model depends on how the model was trained. What makes AI testing special (and even fascinating!) is that engineers generally know how to build or train an AI model but cannot predict the output. The QA teams need to apply new methods and approaches to test this probabilistic logic.

What does all this imply for you? The above points bring up a few questions that you must consider:

Does your team have a “design thinking” mindset?
Is your testing fully optimized and lean?
Is your testing technically focusing on the future and not just the present?
Do you have an insufficient balance between manual QA and test automation?
Do you have a traditional approach to automation?
How can you always see the current proven progress and quality level?

All these are very pertinent queries. Asking these questions can provide high-level pointers to develop your QA readiness for the future. Nagarro’s QA experts can share their insights on such queries and provide a 30,000-foot view of the next quality assurance level integral to every modern enterprise.

Next level QA: What is it?

The next level of quality assurance combines the best of “Design Thinking,” “Mercurial working,” “Hyper-automation for testing” and “Information security” by underlining processes in each of these areas as a mix of being reactive, organized, and adaptive. Let us look at each aspect in detail.

Hyper-automation for testing – Intelligence enabling

How can AI support automation beyond the classical automation of test case execution? We can make QA-related processes quicker, more comprehensive, and more scalable by using intelligent solutions.

Examples are self-healing automation frameworks, intelligent analysis of application logs, automatic root cause-based clustering of failed test cases, and test case writing using AI. Nagarro’s AI4T framework takes test automation to a higher and more intelligent level, immensely boosting your team’s productivity.

Customer experience – Design thinking

Design thinking is about how a customer uses and perceives the product. Their expectations drive customers’ perception of quality based on their experience. User experience is a perception and can be managed by a quality engineering approach that focuses on the end-user.

A user-centric approach can never deliver a functionally buggy system because this would also fail to meet user expectations. User-centric testing ensures that testing is done keeping the end-user experience in mind and that the requirements are validated to meet user expectations. Instead of product functionality, user experience drives testing and test design and prioritizes the overall quality engineering thought process. This approach is also known as total experience QA.

Connected enterprise – Integrated working

Business models are constantly changing to seize new opportunities, supported by new technologies such as:

intelligent devices – sensors are a good example
autonomous components or smart glasses connected via different protocols
performing a business function, such as training via AR, guiding using VR including streaming and transfer to remote guides, experts, or assessors.

Quality engineering of the future will take a dramatic shift to cater to such testing needs.

Cloud adoption – Scalable solutions

On-prem solutions are expensive, less scalable, and complex to maintain. Replicating on-prem infra or environment is also quite expensive. This is where the cloud comes in, providing a cost-effective solution to such bottlenecks. The approach to quality engineering will need to shift by focusing on scalability, availability, and integration with automation solutions.

Besides the above-mentioned changes will become the pillars of the next level of quality assurance, some things will never go out of fashion. It will always be important to remain:

adaptive and lean
responsive to dynamic needs
aligned with fast iterative development and releases.

Creating a lean testing process starts with discovering waste in the current QA and testing lifecycle and eliminating or minimizing them. Anything that does not add value to business/customers and quality is a “waste.” The most obvious example is replicated or indirect communication of topics.

Securing customer data and applications is crucial, especially where data leaks or modifications are unacceptable. Data privacy practices and policies will also be essential to secure customer data and systems. Given the distributed nature of enterprises and their customer interactions nowadays, how can you ensure that the product being developed is secure? Many organizations are already building teams focused on security consulting accompanying the development process, security & penetration testing, and information security teams to fight attacks. Gartner predicts that “Cyber Security mesh” will be among the key technology trends of 2022.

Achieving the next level: How to get there?

The recommendations we discussed are from Nagarro’s 360-degree QA approach. The 360-degree approach is an agile and pragmatic method to deliver measurable improvements in design thinking, faster time-to-market, cost reduction, agility, and efficient IT operational processes, helping businesses achieve quality transformation. Our proven approach has been developed and improved over the last 5 years and has been successfully implemented over 50 times.

Another basis for this approach is the industry’s leading testing maturity & process models TMMI®, TestSPICE, and TPI®. Our 360-degree approach includes:

Comprehensive assessment checklist to evaluate the existing level of quality engineering
Future-of-QA best practice catalog to assess the readiness for short & mid-term horizon’s topics
State-of-the-art model, based on TMMI and TPI to determine the QA organization test maturity level
Recommendations for continuous improvement and raising the level of the test organization
If urgent topics occur, we usually structure the recommendations into two backlogs for a fast lane and sustainable lane

Nagarro is a Gold Sponsor at EuroSTAR 2022. Connect with Nagarro’s experts and learn more about their 360-degree QA approach. Looking to learn more on QA approach? Join us at EuroSTAR in June for lots of talks and sessions on QA, AI engineering technologies, test automation, and so much more.

It’s our first in-person event in 2 years – and it’s going to be a massive celebration of testing! Learn from 70 testing experts, and connect with your peers at Europe’s best testing event. Get your ticket now – book by Feb 28th and save 20% on individual tickets; up to 43% on group bundles. See you in Copenhagen ☀️

Authors

Khimanand Upreti

Khimanand Upreti has more than 18 years of experience in software testing and holds a university degree in Computer Science. Since 2011, Khimanand Upreti has been part of Nagarro and is co-leading the global business unit “Accelerated Quality and Test Engineering,” in short: AQT. His passion for testing is focused on new digitalization topics, such as testing artificial intelligence or quality assurance of chatbots.

Hannes Färberböck

Hannes Färberböck is a Nagarro co-founder and responsible for the global business unit “Accelerated Quality and Test Engineering”, in short: AQT, which he established. In his long career as a consultant, trainer, and entrepreneur, he has done innovative pioneering work in the areas of quality, OOP, architecture, and agile development processes and passionately strives for the highest quality in software projects.

EuroSTAR is celebrating 30 years of software testing

January 26, 2022 by Fiona Nic Dhonnacha

EuroSTAR Conference is celebrating 30 years of software testing this year – that’s three decades of the testing community gathering to spark ideas, learn, and thrive. For 2022, we’re back in-person, in Copenhagen, 7-10 June 2022.

A big thank you goes to the 2022 Programme Committee, who have curated a four-day extravaganza of learning, excellence, and leadership in testing: Programme Chair Graham Freeburn (U.K.), Sue Atkins (U.K.), Morten Hougaard (Denmark), Bart Knaack (Netherlands), and Tone Molyneux (Norway).

Keynotes to inspire

This year’s programme features 5 keynote speakers that are driving testing forward with innovations in their field. Smita Mishra (India) talks about testing software for the next economy. Rikard Edgren (Sweden) shows how to generate a good understanding of testing through continuous learning. Michael Bolton (Canada) shares insights on testing through digital transformation. Nancy Kelln (Canada) explores shifting testing to new methods. Alex Schladebeck (Germany) talks about the future of testing, and how it will shape teams and testers’ work. You’ll walk away with a notebook packed with inspiration.

Dive into expert tutorials

Our 10 interactive tutorial s cover wide ranging topics, from agile to strategy, new technologies, DevOps and lots more. Expert trainers include James Lyndsay, Rik Marselis, Anne Kramer, Gáspár Nagy, Emma Lilliestam & Hanna Schlander. The in-depth sessions will ensure you leave with lots of learnings and actionable outcomes for your work.

Track talks to refresh your learning

Choose from 55 track talks cover everything from testing techniques to test management, skills development, challenges and more. EuroSTAR 2022 track talk speakers include testing legends and superstars Dorothy Graham, Michaël Pilaeten, Ard Kramer, Andrea Jensen, Kristoffer Nordström, and Isabel Evans.

Re-connect with the testing community

Of course, with nonstop networking, there’s plenty of opportunities to strike up conversations with your fellow software testers outside of the talks. Meet new friends over coffee, lunch breaks, and during evening social events. You’ve also got access to the largest software testing expo in Europe! Check out the latest software testing tools and be in with a chance to win some cool swag and prizes…

Join us in celebrating 30 years of software testing: Book by 28^th February and save 20% on single conference tickets. Or, bring your whole testing team and save up to 43%. That’s the best value software testing training you’ll get all year.