Tutorial B

Developing Critical Thinking Skills for Testers

Andrew Brown

09:00-17:00 CEST Tuesday 7th June

We expend huge efforts in training people in various testing skills. Yet, we often fail to train ourselves in the most important testing skill of all– how to think critically about what we are testing. Also, when we test, we are vulnerable to cognitive biases and thinking traps that can catch out even the most seasoned tester.

In this workshop, I share a training course I developed in our organisation, aimed at helping both new and experienced testers develop their critical thinking skills, as well as help address our biases and thinking traps.

One of the most common thinking traps we face is that we tend to think in ways that confirm our existing beliefs, rather than challenging those beliefs. Hence, we begin our tutorial with an exercise to reveal our vulnerability to this bias, then show how we can mitigate our vulnerability.

Another thinking trap is that we often spend too much time collecting additional data about a problem, when we would be better off using that time to generate alternate theories about the problem’s cause. In our next exercise, we diagnose a problem by either collecting additional data or generating alternate theories. We discover that generating alternate theories is more effective.

A further trap is that we create tests that are non-diagnostic, in that their results do not tell us anything new. We use an experiment of selecting different coloured balls from a bag to explore how we create tests that have no diagnostic value, or reach conclusions based on non-diagnostic data.

In another exercise, we explore how our reasoning does not use logic, or at least not rational logic, but rather we use logic based upon social permissions and cheater detection. We use this to explore how to write better tests by rephrasing testing requirements in terms of social logic.

Next, we splash around with some water to explore how we are vulnerable to EINSTELLUNG, or functional fixedness, where our minds solve problems using fixed patterns.

Finally, we explore a problem with 12 weights, to reveal just how much unknown, hidden information may be available within test results.

Come along to this session and learn how you can improve your testing by enhancing your critical thinking skills.