The Peltzman Effect – Why Adding more Test Phases may not Reduce Bugs

Th9     Start Time : 13:30     End Time : 14:00

After a previous release was so buggy, the management team added an additional test phase, expecting quality to go up. However, when it was unchanged, everyone was puzzled and demanded to know why adding more testing had not improved quality.

We test to reduce bugs. We are often tempted to add another test phase or quality process, in the belief that we will somehow reduce the number of bugs in the finished product. But what if adding phases made absolutely no difference to quality?

In this session, we learn of a risk compensation phenomenon known as the Peltzman Effect. We learn why the benefits expected from adding quality processes rarely materialise, but are instead consumed by behavioural changes elsewhere that leave the bug level unchanged. Using examples from traffic safety, dangerous sports, and safety equipment, we show how an increase in safety frequently leads to a compensatory increase in risk taking behaviour.

We demonstrate how this Peltzman Effect applies in software development. We show how improving one area invariably leads to compensatory behaviour elsewhere in the development process, with the net result that quality remains unchanged, even if progress towards the overarching project goal is improved. We explore the circumstances under which the Peltzman Effect becomes important in software development. We learn when adding quality processes improves quality and when adding them makes no difference. We also learn how we can reduce the Peltzman Effect, plus how we can improve quality above its present value in an organisation, not by introducing additional quality phases but by shifting the relative importance of quality as a project goal.

Key Takeaways:

  1. Risk improvements in one area are frequently compensated for by changes in another area. This is the Peltzman Effect.
  2. A quality improvement may result in benefits other than a simple bug reduction. For example, teams may choose to take the benefit as an earlier delivery.
  3. Teams have control over how they choose to consume a quality or other benefit.


Back to Programme

  • Speaker


    Andrew Brown - Consultant, Expleo, United Kingdom

    Dr Andrew Brown is a principal consultant at Expleo. He leads an independent line of research into understanding why we humans make the mistakes that lead to software defects.

    He has 25 years’ experience in the software industry. Previous roles include Head of QA at HMV, Head of QA at a financial software house and a test manager in Japan.

    He holds a degree in Physics and Maths, an MBA from Warwick Business School and a doctorate from Imperial College.

<< previous next >>