2018 was the third installment of the vendor-led case study which featured on the 2018 EuroSTAR Software Testing Conference as part of our Real Customer Case Study. This initiative was specifically designed to give EuroSTAR exhibitors an opportunity to feature on the programme and bring them close to the action. This route is a simplified version of the call for speakers and gives an exhibitor a 1/30 chance of being part of the highly sought after 2018 programme.
Xpand IT were the 2018 winners with their test management tool, XRAY. This tool completely changed the dynamics of their customer Swiss Federal Railways (SBB). To find out more check out their session (which turned out to be one of the most popular sessions at EuroSTAR 2018) below:
About The Tool
Xray is a complete Test Management Tool for managing both manual and automated tests and is seamlessly integrated with Jira. It is a full-featured app that does not require any other software n order to run. It supports the entire testing life cycle: test planning, test specification, test organization in flat or hierarchical way, test execution and test reporting. Xray also supports automated tests that can be run with Cucumber and has integration with other test automation frameworks like JUnit, NUnit or Robot.
With over 4.000 customers in 65 countries, Xray is the fastest growing Test Management app for Jira.
Customer Background
SBB is the backbone of the Swiss public transport system, and day-to-day rail operations are the basis of what they do.
Switzerland opened its federal railway system in 1902. Today, more than 33.000 employees in 140 different professions, along with 600 self-developed and several off-the-shelf applications work together to operate an internationally integrated railway network. Despite the size, SBB maintains the world’s most accurate arrival and departure schedule. The Swiss railway system is a model for other railways around the globe.
Customer Challenge
SBB used HP ALM for 12 years. It had more than 100.000 test cases, many more test runs and 1.000+ trained employees helped ensure that operations stay on track. However, users started to boycott the tool due to the lack of integration in SBB’s existing toolchain and because of all the communication problems that arouse around using other terms for the same things.”
Besides the general IT problems of a big company like SBB, software testers were facing the typical challenges of IT departments that grew too fast and were forced to have management overhead for controlling these many employees. Strictly sticking to IEEE829, they first had to write a test plan for that specific project or team, quickly realizing the master test plan or overall test strategy was not helping or even forcing them to spend time in ways that would not benefit neither the project during its run time nor the company on the long run.
The Approach & Solution
Since SBB’s Agile Transformation, several kinds of experiments are allowed without manager approval, as long as you stay within certain constraints, so they started to search for a test management tool that would improve the collaboration within the team and be less expensive.
But before buying a new tool, the stakeholders made their interests clear:
a) Reduce number of floating user licenses in old tool to gain money for the new tool
b) Protect the already existing test cases and test results
c) provide a stable and easy process on how to train users on the new tool and how to support them
So, after calculating business cases and setting up internal online documentation and support contacts, SBB purchased an XRAY unlimited company license.
By using Xray, SBB was able to work with Jira in all their test management processes. The tool allowed testers to manage all their tests and executions as Jira entities with all the power of the customized screens, fields and workflows, not just regarding the manual testing but also the automated testing. It also had an included REST API that would help SBB in their current and future integrations with several testing frameworks used in load tests, Functional Testing, UI Testing, security testing and others, as well a direct integration with CI tools like Jenkins and Bamboo.
Reporting and Requirement Traceability was something very important for SBB and now test managers and project managers are able to check the test coverage status directly from the issue screen in Jira as well as having a dedicated Requirement Traceability Matrix Report.
With more than 100.000 test cases, XRAY offered the ability, not just to organize tests in a flat way for those teams who would like a more agile like approach, but also to organize them in the Test Repository with tree-like organization at the project level. This way users can hierarchically organize Tests within folders and sub-folders, similar to what was found in older tools. Regarding the planning executions, Xray offered the ability to create test plans for tracking a set of tests and planned or ad hoc test executions; all this with the possibility to have consolidated results.
XRAY helped Swiss Federal Railways move 25.000+ test cases from HP ALM in just 6 months, and paid for itself in only 3 months. XRAY has cutting-edge UX and integrations into the rest of the development toolchain. Now, even teams without a prior test management tool come to XRAY.
Takeaways
With the new tool many things came unexpected, but luckily for SBB it went far better than expected.
“Give the teams the tools they need, the tools that enable collaboration, ease human-to-human communication and help decrease misunderstandings. When team members have more transparency, they develop more respect for the work of others and therefore have a better understanding what really drives the other persons in the team”, says Andreas Wieczorek, Senior Test Manager and Team Leader at SBB.
What really impressed SBB the most was that all the different specialists outside the teams, normally refusing to even read test cases, are now eager to catch a test case that is freshly written or even write it by themselves. “Those people love test cases, and what drove them away was a tool lacking every collaboration feature.”
Team members now freely contribute to the growing user number of XRAY: be it translations, finding solutions for issues other teams have or helping them to set it up or even organize calls with the supplier and do internal know how transfer meetings.
“For me personally, the agile movement within SBB plays a big part of it, but the integration of new tools that put collaboration as first priority is equally necessary”, notes Andreas.