Programme launch offer - Save 20% Book Now

EuroSTAR Conference

Europe's Largest Quality Engineering Conference

Track Talk W12

The Anomalous Tester

James Bornefelt Westfall

15:00 - 15:45 CEST Wednesday 4th June

Meteorites struck Oslo, Norway in 2012. One of them crashed through the roof of someone’s garden shack. They did not make any automatic tests fail at my workplace. But they could have.

With so many variables potentially affecting the outcomes of automated tests, how do we see clearly? The answer is data: time series of test result data over time. How do we bring in machine learning to see even more clearly? The answer once again is data: well-structured historical test data for training the models.

This talk will cover an “Aha” insight that came to me while testing the Norwegian Ministry of Education’s system for national examinations. I was shocked by the random things happening when we started collecting logs during exam periods when up to 70 000 students were simultaneously connected to the system.

This experience impressed upon the need to capture data and build systems that cut through test result data noise and isolate meaningful, actionable trends. Trends that point to either latent problems in test infrastructure or to real problems in the system under test.

While the majority of test failures when taken individually are simple and unambiguous, reliably catching all of the insights you could have and should have captured from your test runs before a release is an entirely different problem.

I will demonstrate in detail how we calculate meaningful trends to give actionable insights. This will cover how we work strategically to fill holes in the data we collect to uncover latent problems that frustrate our customers. Finally, I will demonstrate how we use machine learning with homegrown models to solve the thorny problem of what to do with tests that we expect to fail.

It will be a wild ride. But I promise practical recipes you can apply to tame the chaos (somewhat). But still no solution for the meteorites.

What you will learn

  1. The benefits of looking at automatic test trends over time instead of just focusing on the last results
  2. How to capture test results as time series and how to turn that data into actionable insights
  3. How to use machine learning to solve the resolve the issue of tests that are expected to fail in a less problematic way than just disabling them

Session Details

  • Intermediate
  • 45 minutes
  • Includes 15 mins Q&A
  • Efficiencies and new capabilities
Buy Conference Ticket

Check out this video from James Bornefelt Westfall to hear more about his talk, ‘The Anomalous Tester.’

We look forward to welcoming you to EuroSTAR 2025.

Session Speaker

James Bornefelt Westfall

James Bornefelt Westfall

Neatframe AS, Norway

James has been working for over 20 years in software and hardware development. He has worked with test automation since 2008 and has gradually switched from a successful developer and architect role over to an experienced test automation specialist role.

He focuses on how automatic end-to-end testing can bring together all the participants in an agile team and deliver value early in the requirements gathering and active development stages, long before the software is close to being finished.

When not giving Neat’s video-conference solutions strange dreams, James enjoys the Norwegian outdoors, especially when skiing and fly-fishing.

BACK TO PROGRAMME

Ticket Offer

Save 20% on individual tickets until February 28th

Book a group and save even more. The more you bring, the more you save.

Individual

From €1,352

save 20%

BOOK TICKET

Groups

From €1,002

Based on 10 Tickets

Book Group

Stay in the Loop

We want to ensure you never miss important announcements, updates and special offers from EuroSTAR.