Programme Launch Offer: Save 20% - Book Now

EuroSTAR Conference

Europe's Largest Software Testing Conference.

Track Talk, T17

Metrics That Matter for GenAI Evaluation

Anamika Mukhopadhyay

Deepshikha

15:45 - 16:30 CEST, Tuesday 16th June

As Generative AI becomes part of everyday life, organizations are embedding LLMs and AI agents into mission-critical workflows. Yet most AI teams still rely on traditional metrics like accuracy, precision, and recall, which are useful for many AI systems but insufficient for generative tasks. Generative AI does not predict right or wrong; it creates possibilities. How do you evaluate creativity, coherence, safety, fairness, or trustworthiness with these metrics alone?

In this talk, we share our experience evaluating LLMs and AI agents across diverse industries and how we designed KPI frameworks tailored to specific contexts. Key insights include:

* Why conventional metrics alone cannot capture the complexity of generative AI
* New dimensions of measurement, from reasoning and factual consistency to fairness and robustness
* The necessity of use-case-driven evaluation for chatbots, summarizers, and autonomous agents
* Designing KPI frameworks that balance technical performance with business impact
* Practical lessons from applying these strategies in real-world deployments

To build Gen AI systems that are reliable, trustworthy, and valuable, we must measure more than correctness. This talk will show you how to define what good looks like, not just in numbers but in outcomes that truly matter.

What you will learn

  1. Understand why traditional metrics such as accuracy, precision, and recall are not enough for evaluating generative AI
  2. Explore new dimensions of evaluation, including reasoning, factual consistency, fairness, robustness, and alignment with business goals
  3. Discover actionable strategies for designing KPI frameworks that balance technical performance with business impact

Session Details

  • Intermediate
  • 45mins
  • Includes 15min Q&A
  • Testing the reliability, fairness, and safety of AI models
Buy Conference Ticket

Check out this video from Anamika Mukhopadhyay and Deepshikha to hear more about their talk on Metrics That Matter for GenAI Evaluation.

We look forward to welcoming you to EuroSTAR 2026.

Session Speakers

Anamika Mukhopadhyay

Nagarro, India

Anamika, a QA consultant with 10+ years in software testing and automation, leads Nagarro’s Emerging and Special Tech Practice. Passionate about inclusivity and equality, she explores AI-driven innovations to advance testing. Her focus is on delivering sustainable, accessible, high-quality solutions that empower diverse users across the evolving technology landscape.

Co Session Speaker

Deepshikha

Nagarro, India

Deepshikha, with 10+ years in software testing and quality engineering, heads Nagarro’s AI in Testing practice. She drives sustainable, AI-powered automation strategies that boost efficiency, optimize processes, and speed up delivery. Her expertise empowers organizations to achieve reliable, high-quality software while enhancing business value, performance, and long-term impact.

Programme Launch Offer

Save 20% on individual tickets until February 28th!

The more you bring, the more you save.

Individual

From €1,840

3 Day Ticket*

BOOK TICKET

Groups

From €1,380

Based on 10 Tickets

Book Group

Stay in The Loop

Subscribe to our newsletter and never miss important announcements, updates and special offers from EuroSTAR.

part of the