Year End Offer - Save up to 50% on tickets - Book Now

EuroSTAR Conference

Europe's Largest Quality Engineering Conference

Track Talk T7

Beyond the Wow – Lessons of LLM-aided Test Case Generation

Theodor Hartmann

12:00 - 12:45 CEST Thursday 5th June

In the realm of software testing, the use of Large Language Models (LLMs) for test case generation has sparked both interest and skepticism among professionals. In my talk, “Beyond the Wow – Lessons of LLM-assisted Test Case Generation,” I will share practical insights gained from developing a tool that combines LLMs with a basic Retrieval-Augmented Generation (RAG) approach. This tool generates test artifacts such as test cases and test data by utilizing common knowledge alongside customer-specific documentation, requirements, and user stories. I will discuss the complexities and challenges encountered in implementing and applying LLMs in testing.

This session will focus on the realities of using LLMs in test case generation, specifically addressing challenges that often get overlooked by those outside the testing field. I will cover common issues such as redundancy in generated content, the difficulty of achieving consistent results, and the varying effectiveness of LLMs depending on the type of application. Additionally, I will explore the contrasting experiences many testers face: the “fear of the blank page,” where the pressure to create quality test cases can be daunting, versus the overwhelming nature of generated content that may be verbose, but sometimes nonsensical. I will illustrate the field of tension mentioned above with examples from experience.

In a landscape where non-testers often believe that all aspects of testing can be automated, I will equip testers with compelling arguments to explain why a cautious approach is essential. These points will emphasize the importance of human oversight, the need for contextually relevant outputs, and point at the inherent limitations of LLMs that challenge the notion of complete automation.

What you will learn

  1. Practical Evaluation: Learn to critically assess LLM outputs and identify when they provide real value versus when they may introduce confusion or redundancy
  2. Effective Integration: Discover strategies for effectively incorporating LLM-generated content into testing workflows while maintaining high standards of quality assurance
  3. Communication Skills: Equip yourself with key arguments to address questions from non-testers about the cautious approach to adopting AI in testing, emphasizing the importance of context and human oversight

Session Details

  • Intermediate
  • 45 minutes
  • Includes 15 mins Q&A
  • Implementing/integrating AI tools
Buy Conference Ticket

Check out this video from Theodor Hartmann to hear more about his talk, ‘Beyond the Wow – Lessons of LLM-aided Test Case Generation’.

We look forward to welcoming you to EuroSTAR 2025.

Session Speaker

Theodor Hartmann

OBJENTIS Software Integration GmbH, Austria

Theodor Hartmann began his journey in software testing in 2000 as an intern. Over the past 20 years, he has gained extensive experience across various industries, including insurance, telecommunications, and banking. With a passion for the technical aspects of testing, he enjoys uncovering defects and exploring the philosophical questions surrounding the purpose of testing, while staying curious about the constants in testing amid the evolving landscape of new technologies. Currently a Senior Consultant at OBJENTIS, he plays a key role in developing testing tools, a position where it’s sometimes harder to embrace found defects.

BACK TO PROGRAMME

Year End Saver

Save 30% on individual tickets until December 31st.

Book a group and save even more. Teams save 50% – the more you bring, the more you save.

Individual

From €1,610

save 30%

BOOK TICKET

Groups

From €1,176

Based on 10 Tickets

Book Group

Stay in the Loop

We want to ensure you never miss important announcements, updates and special offers from EuroSTAR.