Year End Offer: Save 30% + Teams Save Up to an Extra 25%!

EuroSTAR Conference

Europe's Largest Quality Engineering Conference

Track Talk, W15

Using AI to Create UAT Tests, a Case Study from BBC Radio

Bill Watson

14:45 - 15:30 CEST, Wednesday 17th June

AI seems to be everywhere, the technological zeitgeist, but just how effective is it in generating test cases? I wanted to see if we could come up with some actual data on this, but being able to run different tests on the same “fresh” software was problematic.

However, this changed with a project to upgrade the pan BBC radio playout system, used in multiple locations across the UK. This means some 350 new test cases need to be written and then tested at different radio stations with different hardware configurations.

This allows us to set up a study with three different sets of tests then run them concurrently: two sets of AI-generated tests and one control set using our current methodology.

As we will be running these tests multiple times in different locations, we can now create a study and compare different KPIs to see what AI can and cannot add to our test case creation process. KPIs being measured include time to create test cases, time to run tests, and defects found (and not found) by each technique.

At the end of the project, we will have metrics on what has worked best in terms of effort of input against goals achieved, as well as having fully explored how to best use AI for manual test creation.

This talk will cover the methodology we are using, including how we decided what metrics to focus on, the results achieved, and details on what we establish as best practice for creating UAT tests with (or without?) AI assistance.

This is an ongoing study, so at submission time we cannot predict what the results will be. Whatever they are, the journey to getting them is the key, laying out an approach to evaluating how AI can work in testing.

What you will learn

  1. Framework to use to evaluate how and if AI test case creation fits in their organisation
  2. What approach to take when implementing the roll out of UAT tests in an AI world
  3. Be able to use AI to assist SMEs in generating UAT test cases

Session Details

  • Introductory
  • 45mins
  • Includes 15min Q&A
  • Testing the reliability, fairness, and safety of AI models
Buy Conference Ticket

Check out this video from Bill Watson to hear more about his talk on Using AI to create UAT tests, a case study from BBC radio.

We look forward to welcoming you to EuroSTAR 2026.

Session Speakers

Bill Watson

Bill Watson

Test Manager – BBC

My first foray into software came with a Dragon 32 computer in the early 1980s. Having a computer with a less than popular OS meant adapting programmes from a ZX Spectrum magazine and lengthy debugging sessions!  After diversions into a number of non-tech fields, I returned to the fold and now have two decades experience in the IT industry.

I’ve worked in both contract and permanent roles in Europe and Africa ranging in scale from multinationals, to being the first tester in the door at small companies. I’m currently a Test Manager in the BBC, testing outsourced IT projects..

Ticket Offer

Book a group and save up to 30% on tickets!

The more you bring, the more you save.

Individual

From €1,610

3 Day Ticket*

BOOK TICKET

Groups

From €1,208

Based on 10 Tickets

Book Group

Stay in The Loop

Subscribe to our newsletter and never miss important announcements, updates and special offers from EuroSTAR.