Everything I Know About Testing I Learned from the Scientific Method

W2     Start Time : 09:45     End Time : 10:30

We’re told that Software Development is an Engineering activity. But I was trained as a Zoologist and I have always found that testing is more like the discovery of Science than the predictability of Engineering. Every day working as a tester I use aspects of the Scientific Method. The most significant part of this is in Experimental Design, which taught me virtually everything I needed to become a tester and has kept me going for 25 years.

Science properly begins with a Hypothesis – this is a set of testable statements about a phenomenon that make explicit predictions about its behaviour under specific circumstances. Sounds familiar? That’s because it is similar to Good Requirements (you may have heard of those mythical beasts).

Scientists design experiments to test the hypothesis. Good experiments are not those that will “prove it”, but those that will demonstrate it is false. Experimental design must consider alternative explanations for the predicted behaviour, other factors which may influence the result and must specify initial state, inputs and the outcomes in unambiguous terms. Experiments that fail to control their environment are almost certain to fail.

In an ideal world, well-designed experiments run in sufficient quantities to provide good coverage, and which fail to disprove the hypothesis result in support for it. In Science, a hypothesis is never proven, and is always provisional. In Science, as in Software, there is always the possibility of the test not run that would uncover the fatal flaw. Science addresses this with detailed statistics on probability, where we generally run with an acceptance of some level of risk.

A Hypothesis that has been well tested and has not failed may become a Theory – this is as good as it gets. Software in the same situation becomes “product” which we launch or put on sale.

In summary, Software Testing has a lot to learn from the Scientific Method, and this aims to be a light-hearted run through the parallels.

Want to attend? Book your Conference Place

  • Speaker


    Paul Coyne - Test Architect, Clydesdale Bank plc, Scotland

    I started working, reluctantly, as a tester in 1989. Ever since then I have desperately been trying to find another job that’s half as interesting. The only thing that came close was a year out spent fixing up an old house in France.

    I’ve worked in various organisations in multiple levels and roles, though mainly in finance and utilities. I spent twelve years as a self-employed consultant.

    One of the reasons I love this job is that I really care about doing things right:- not perfection – that’s a distracting myth. But “good enough” is rare enough to need cherished!