Year End Offer - Save up to 50% on tickets - Book Now

EuroSTAR Conference

Europe's Largest Quality Engineering Conference

Track Talk W5

Unmasking AI: Is Artificial Intelligence Truly Thinking?

Benjamin Johnson Ward

11:30 - 12:15 CEST Wednesday 4th June

As AI, especially modern large language models, integrates into critical aspects of our lives, a pressing question arises: Are these systems truly reasoning, or merely performing pattern matching? This talk examines the nature of AI decision-making and its implications for quality and robustness.

We’ll consider reasoning from a human perspective, contrasting it with current AI models. Tracing AI’s evolution from symbolic logic to today’s LLMs, highlighting the shift from rule-based reasoning to statistical models based on data. Notably, research suggests that while LLMs can emulate reasoning patterns, they may lack genuine understanding—a concept known as “syntactic manipulation without semantic comprehension.”

But does this distinction matter? Is an AI’s lack of true reasoning fundamentally different from human reasoning errors? We’ll explore how these differences impact the quality and fairness of AI systems, examining studies and methods to assess AI reasoning and implications for applications.

One critical factor we will consider, is training data contamination, which can cause AI models to “cheat” by memorizing answers instead of reasoning- misleading us into believing they possess deeper understanding. This directly impacts the quality and trustworthiness of AI outputs.

We’ll delve into practical approaches like adversarial testing, counterfactual evaluations, and detecting training data contamination, to uncover biases, model drift, and weaknesses in robustness.

With AI technology rapidly evolving, we’ll consider how future advancements toward human-like reasoning could impact testing and quality assurance practices.

Attendees will learn the key differences between human reasoning and AI’s pattern recognition, and their effect on application quality and fairness. They will gain practical testing approaches to assess AI quality—bias, drift, robustness—and detect training data contamination, evaluating if the absence of true reasoning affects risks. Moreover, they will learn how to refine testing strategies to remain effective as AI technologies evolve.

What you will learn

  1. Acquire practical testing approaches—such as adversarial testing, counterfactual evaluations, and detecting training data contamination—to assess the quality of your AI applications, including bias, drift, and robustness.
  2. Understand the key differences between human reasoning and AI's pattern recognition, and how these differences impact the quality and fairness of your AI applications.
  3. Develop strategies to refine and adapt your testing practices, ensuring they remain effective as your AI technologies evolve toward more sophisticated reasoning capabilities.

Session Details

  • Intermediate
  • 45 minutes
  • Includes 15 mins Q&A
  • Testing the reliability, fairness, and safety of AI models
Buy Conference Ticket

Check out this video from Benjamin Johnson Ward to hear more about his talk, ‘Unmasking AI: Is Artificial Intelligence Truly Thinking?’.

We look forward to welcoming you to EuroSTAR 2025.

Session Speaker

Headshot of EuroSTAR 2024 speaker Ben Johnson-Ward

Benjamin Johnson Ward

Curiosity Software, US

Ben has over a decade of experience pioneering innovative testing techniques and tools in the software industry. With a background in behavioural economics from LSE, he explores how unconscious biases influence behaviour and critical thinking in testing. Throughout his career, Ben has applied this introspective mindset across various quality roles—including developer, product manager, and tester—working with both start-ups and multinational corporations. He’s helped shape testing techniques at global organisations and is passionate about model-based testing, test data, and recently, AI- especially how we can understand AI systems themselves, and explore new frontiers in AI-driven testing.

BACK TO PROGRAMME

Year End Saver

Save 30% on individual tickets until December 31st.

Book a group and save even more. Teams save 50% – the more you bring, the more you save.

Individual

From €1,610

save 30%

BOOK TICKET

Groups

From €1,176

Based on 10 Tickets

Book Group

Stay in the Loop

We want to ensure you never miss important announcements, updates and special offers from EuroSTAR.