Track Talk F10

How Aware Are We of Our Biases in Testing?

Alexandru Cusmaru

Eduard Enoiu

10:00-10:45 CEST Friday 10th June

Testers often struggle, not only with the technical challenges of the test environment but also with more abstract questions like: “Why did I not find that bug earlier?”, “Why did I not think of it?” or “Why do I do not find the important bugs?”. Answers to these questions can be single or multiple, simple or complex.

One of the answers can be: “Your testing is biased!”. And yes, bias is inevitable in all of us and yes, it is OK!

Unconscious biases are hard-wired behaviors that influence testers and can set them on an incorrect course of action. If we understand how these biases affect testers’ everyday behavior, we can attempt to mitigate them, then develop more effective tools and strategies to help testers avoid the pitfalls of cognitive biases. But to what extent are testers aware that cognitive biases influence their work? This was the primary question we sought to answer in our research. We developed a questionnaire survey, designed to reveal the extent of software testers’ knowledge about cognitive biases, plus their awareness of the influence biases have on testing. We reached out to software professionals working in different environments and gathered valid data from approximately 60 practitioners.

Our results suggest that professionals are indeed aware of biases. Specifically, they are aware of preconceptions such as confirmation bias, fixation, and convenience. In addition, optimism, ownership, and blissful ignorance were other common biases. In common with other research, we observed that people tend to identify more cognitive biases in others than in their own judgments and actions, indicating a vulnerability to bias blind spot.

In our session we will present our results, then involve the audience in a discussion of the interpretation of those results. Specifically:

  • Our reasoning behind the questions
  • Demographic data of the participants
  • Helpful practices when confronted with biases
  • The results and potential conclusions
  • If the question of awareness has been answered or not?
  • Next steps for our research