Programme Launch Offer: Save 20% - Book Now

EuroSTAR Conference

Europe's Largest Quality Engineering Conference

Track Talk, W8

Can We Measure Local Bias in AI Systems?

Wessel Braakman

Teresa Herland

12:00 - 12:45 CEST, Wednesday 17th June

AI bias is a well-known problem, but most studies have been conducted by institutions in the United States and focus on the English language. But what if we use such systems to make (partial) decisions that affect people in Norway, where we live? Can we investigate the same issues in Norwegian and in the context of Norwegian society?

We are three colleagues who worked on this research besides our day-job, because we feel like this is an important part of AI development and quality assurance. We have, since we started, set up our very own non-profit association in which we can do this research company-independent.

In this deep dive, we would like to share our findings from testing AI models for bias in the Norwegian language and society. We will share what worked, what didn’t, and what we learned along the way. The research we do is aimed at the Norwegian language and society, but can of course be mapped to other countries and cultures. What we provide is a clear guide on how to achieve this, including all we learned.

As a foundation, we use research from BBQ: A Hand-Built Bias Benchmark for Question Answering, which we adapted for the Norwegian context. We will explain about how we prepared our data, how we got responses from AI systems, and how we then scored them to find bias. We will also talk about automation, what worked and what didn’t.

We will walk through our method, the challenges we faced, and why this is important for AI development in Norway (and in other countries). You as a listener will gain concrete insights into how to evaluate and reduce bias in AI models so that they work fairly across languages and cultures.

This session will give you the tools to think critically about AI fairness, beyond the usual English-centered approach.

What you will learn

  1. What is bias in AI and why do we need to make it visible
  2. How to use/adapt a research method to find bias in your local language and society
  3. How biased are the AI systems we researched

Session Details

  • Introductory
  • 45mins
  • Includes 15min Q&A
  • Testing the reliability, fairness, and safety of AI models
Buy Conference Ticket

Check out this video from Wessel Braakman and Teresa Herland to hear more about their talk on Can We Measure Local Bias in AI Systems?

We look forward to welcoming you to EuroSTAR 2026.

Session Speakers

Wessel Braakman

Bouvet ASA, Norway

Wessel is a skilled expert in the field of test automation, with over 12.5 years of experience in the industry. His strong interest in AI systems focuses on understanding their technical foundations and developing methods to critically assess their quality; particularly the softer aspects that affect user experience and ethical implications.

Session Co-Speaker

Teresa Herland

Rema1000, Norway

Teresa has over 10 years of experience as a test and quality manager. As a quality manager, she is concerned with the quality of the AI models themselves. How does the choice of AI system affect the results we get, how do we ensure they are “fit for purpose”?

BACK TO PROGRAMME

Programme Launch Offer

Save 20% on individual tickets until February 28th!

The more you bring, the more you save.

Individual

From €1,840

3 Day Ticket*

BOOK TICKET

Groups

From €1,380

Based on 10 Tickets

Book Group

Stay in The Loop

Subscribe to our newsletter and never miss important announcements, updates and special offers from EuroSTAR.