Track Talk F5

How to Avoid Cognitive Biases in Testing

Benjamin Johnson-Ward

10:30-11:30 Friday 14th June

We can’t test everything. Instead, testers must think critically and creatively, making risk-based decisions about what to explore, and what to release “on trust”.

But, what if this ability to think can lead us astray? What if our brains have evolved to evaluate risks in ways that can produce suboptimal decisions? Decisions that appear rational might in turn focus finite testing efforts on areas of the system where bugs are less likely to occur. This then overlooks at-risk areas where impactful defects can hide.

In this talk, I will teach you about “cognitive biases”, and how to avoid them in your testing. I will draw on Daniel Kahneman and Amos Tversky’s pioneering research in behavioural economics and cognitive psychology, considering how our brains have evolved to behave irrationally. You will learn about “heuristics”, which provide shortcuts for the brain to make fast, in-the-moment decision, but can also undermine slow and considered thought.

A series of interactive questions and examples will then teach you how cognitive biases are at play in your own thought. This will show you how cognitive biases can degrade your testing, from the “Positive Test” and “Confirmation” biases, to the “Availability Heuristic” and “IKEA Effect”.

I will end with suggestions for combatting cognitive biases in testing. I will discuss the inclusion of a wider range of stakeholders in testing decisions, the use of static code analysis and data-driven metrics, and the formulation of tighter specifications to invite critique.

The session is designed to be fun, educational, and useful for building better testing teams/practices. In terms of the conference theme, you will learn how “what we are doing here” might not always be what we *think* we are doing. Better risk assessment then makes testing more “relevant, powerful, and valuable” in “particular situations and contexts”.