go back to the blog

Real Customer Case Study Winner 2017: Adaptavist

  • 22/03/2018
  • no comments
  • Posted by Paul

2017 was the second year a vendor-led case study was featured on the EuroSTAR Software Testing Conference Programme as part of our Real Customer Case Studies competition. This initiative is designed bring the Expo vendors closer to the action with a simplified route to the conference programme rather than relying entirely on the rigorous programme selection process which has traditionally proven a difficult path to navigate for vendors.

Look back at last year’s winners Adaptavist & Loblaw Digital as they present their case study at EuroSTAR 2017 in Copenhagen, Denmark. Details on the case study are also provided below.



About The Vendor

Adaptavist helps enterprises continuously develop, deliver, and manage their software better using Atlassian tools. We are a global consultancy and an Atlassian Platinum Solution Partner for the Enterprise. Our clients are among the most complex organisations in the world and include over half of the Fortune 500. To help them get the most from their application lifecycle and we’ve created – or evolved – an organisation that is solid in its purpose but flexible in its execution.

Adaptavist has a strong track record in developing Atlassian Apps. Not only is Adaptavist the creator of the fourth top selling app on the Atlassian Marketplace (ScriptRunner for Jira), but our software has also been incorporated into the core of Atlassian products. Adaptavist Test Management (ATM) is the Enterprise-grade test solution for Jira and one of the 20 Adaptavist apps available in the Atlassian Marketplace. Seamlessly integrated into the Jira user interface, it’s flexible, intuitive and powerful. There’s no need to switch between environments, you can do everything right inside Jira.

Customer Background

Loblaw Digital is an Agile digital agency built to service the online needs of Loblaw Incorporated, Canada’s largest retailer. Spanning four separate brands, Loblaw Digital is responsible for creating and delivering omni-channel experiences across physical and digital channels, including online grocery offerings, traditional e-commerce, loyalty, financial services, and pharmacy products. Brands include: Loblaws Click & Collect for groceries, the award-winning Joe Fresh site for clothing, for beauty products, Shoppers Drug Mart for prescription medications and the loyalty programmes PC Plus and Shoppers Optimum.

The team at Loblaw Digital is driven by a tireless curiosity about where the world of digital products and services are headed and are excited to help shape that future.

Customer Challenge

Bug free and continuously updated retail websites are vital to Loblaw’s business and creating happy customers. However, managing so many high-traffic digital properties with numerous updates per week presented a resource intense challenge for Loblaw Digital’s testing team. Whenever a single test needed to be run, it would have to be written manually and each test then executed individually. The ability to scale and execute tests, instantly generate traceable artefacts (bugs/reports/logs) that are easily actionable by anyone on the TE/Development teams.

Loblaw Digital needed systems and processes that could manage, automate, and interpret the results. They were looking for a solution that provided automated testing, as well as the processes that would remove bottlenecks and scale testing to meet the demands of a large-scale Agile development team. In addition, they wanted a testing solution that was tightly integrated with Jira, the development team’s central planning tool across their entire development lifecycle.

In evaluating the options, Loblaw Digital reviewed many Jira-compatible test management software solutions and, for nearly two years, failed to find something to meet their team’s needs. In terms of integrated automation, feature set and extensibility, nothing they found could deliver what was required.

The Approach

Adaptavist Test Management (ATM) is a full-featured, enterprise-ready test management solution that enables organisations to centralise the entire testing process directly inside Jira. ATM offers a full traceability of requirements, folder hierarchies for test libraries, test case versioning, test data parameterisation, shared steps for reusable common actions, a free and flexible REST API, and the ability to automate bulk tests and CI tools.

By implementing ATM across Loblaw Digital’s Jira instance, the team were able to seriously scale up its testing, by utilising permutations. The idea being that every test case has a number of permutations. In Loblaw’s case – given the number of possible browsers, languages, devices, and user agents that might be involved, multiplied by Loblaw’s different brands – the number of permutations to be tested could be up to 366.


Loblaw Digital started seeing increased productivity and product velocity, and a reduction in risk almost immediately after implementing Adaptavist Test Management. Specifically, they found they could extend and customise ATM in ways that fostered innovation across their team.

“We found that no other piece of test software on the Jira side had figured this out or truly understood how to do that elegantly. We are able to get results in faster than we could imagine by using the API to populate all the results in environments and we are able to deliver high quality feedback to developers and management on the status of a sprint. The coverage of testing has drastically increased. Automation has unlocked extra potential for what we can actually test.”

Loblaw Digital is now running around 250,000 tests a day, streamlining the delivery of new features. Results from tests are uploaded in real time, meaning issues can be detected more quickly. Coverage is also more comprehensive with ATM, meaning more issues come to light. Developers and Testers are able to closely collaborate over the course of feature implementation, allowing for continuous improvement, resulting in higher quality code.The traceability provided by ATM’s reporting engine ensures that no untested ticket ever goes out and nothing gets missed.

“It makes a great case for automation,” explained Justin Commu, Loblaw Digital’s Senior Manager “but an even better case for ATM because if I want to run, say, a login test and I want to say I have one hundred percent coverage, we only need to write and save that test once. It has just one set of steps and we can use environments to see how that test is running in all 366 permutations.

Another key benefit of ATM is a reduced regression period. Regression is the pause before a product is released which is designed to check that it is fully tested, against every permutation, and that changes and compatibility are tested too. Regression is now 2.5X times faster. What’s more, Loblaw Digital believes in the future they will be able to make regression five times faster with ATM.

Takeaways for the Customer

Justin Commu, Loblaw Digital’s Senior Manager, characterises ATM as ‘a fresh take on testing.’ “Not only does it feel tailored for the testing process, it has clearly been intelligently designed,” he says. “All Loblaw Digital’s initial criteria were met. In addition, we finally had a solution where developers and test engineers, had a sensible way of looking at the tests. Moreover, using the Adaptavist API, we could get test results uploaded and stored programmatically without human intervention.

Straight away, we started seeing features that would drive increased productivity and an increased ability to get features to production more quickly and with less risk. Specifically, we found we could extend Adaptavist Test Managment in ways that would allow us to mould it to our specific needs.

ATM is giving Loblaw Digital end-to-end automated testing which Justin Commu describes as “a truly first-class test management experience.” At the same time, it’s not an insult to say that Loblaw Digital’s people barely know it’s there. “That’s fantastic,” notes Commu. “Honestly, the best test management and test reporting tool that I can imagine is the one that I don’t realise I’m using.”

To enter the Real Customer Case Studies Competition for 2018 please click the button below:


Register! Find Out More

Blog post by

go back to the blog


Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery