The Unexpected Joys of Beta Groups

Beta groups are a joy for lazy test managers, letting the end-users do the testwork is a fascinating approach. Beta groups are a joy for lazy test managers, letting the end-users do the testwork is a fascinating approach. RuterReise is an app for android and iOS that allows people to plan their public transport on trams, metro, buses and boats in Oslo and surrounding counties.

The app has more than 100.000 unique users on average.We decided to create a beta group for RuterReise. The idea initially came from the ambition to detect bugs that we couldn’t find ourselves, and to get good test coverage for different combinations of devices and OS versions with low cost and low effort. In addition we were able to explore the users use of the app and getting a lot of the cornercases on the table. These cornercases would never have been found by testing in lab environment!

To enable the users to send reports and chat with us, we connected the product Instabug to the app.. This gave us instant feedback from customers and the possibility to interact with them.
After a while we realized that we could use the beta group to achieve more benefits than just bug detection. E.g. we can:• polls about what the user thinks of specific functions.• Instant feeback on changes• Rapidly develop new features, do a quick test, and let the Beta group do its work

A Betagroup gives us the benefits of shorter test cycles and thus reduced time to market for new features, and the ability to better predict what features will be appreciated by the users.In addition, we found that our users where very keen to tell us what they thought where wrong. Not only with itineraries, but also what was wrong with the services from Ruter in general.

My team decided to use this insight and try to make improvements, not only to the app itself, but to the back-end systems that the app uses.We are also able to provide informal feedback to the beta users, and can ask them in more depth about what they really think is wrong. That proved to give us great insight in the traveller’s mind.

  • Speaker

  • Haavard Gulbrandsen - Test Manager, Promis Qualify, Norway

    I have been a test manager for a number of years, the last 5-6 years in the area of app testing.

    I have been a test manager for is NRK TV, NRK Super, RuterBillett, Ruterreise.

    I have always been keen on finding news ways to test an app both manually and automatic!