Why is my Regression Pack full of Junk Tests? The Endowment Effect

T3     Start Time : 15:15     End Time : 16:00

Talk Abstract:

Regression packs, both manual and automated, have grown in importance, particularly with the advent of Agile, DevOps, CI, CT, and CD. It therefore makes sense to invest effort in populating our regression packs with high-value tests, then maintaining and grooming them, so that they always contain the highest possible value tests.

Unfortunately, regression packs are frequently neglected. An examination of the typical regression pack will reveal it to be full of low value tests that offer little or no benefit in running, whilst at the same time, newer and better tests have been created but cannot make it into an already bloated pack. Even when we are aware of this problem, we seem unable to do anything about it. Why do we allow regression packs drift into this state? A major reason for this drift is a cognitive bias, known as the Endowment Effect. This bias causes us to place more value on an item simply because we own it.

In this session, audience members will participate in a practical demonstration, where they discover how difficult it is to give something up, even after only the briefest period of ownership. We will then explore further how this endowment effect causes us to resist replacing existing tests with new ones, thus causing our regression packs to gradually degrade in effectiveness. We also look at potential approaches to mitigating this problem, together with examining other areas of software development and testing where we are vulnerable to the endowment effect. If you have ever wondered why your regression packs have become bloated and ineffectual, come to this session and learn how the endowment effect distorts your decision-making around regression packs and other assets.

Key Takeaways:

  1. Regression packs are important but frequently full of low value tests, due to a cognitive bias known as the endowment effect.
  2. We can improve our regression packs by reviewing the contents whilst aware of the distortions caused by the endowment effect
  3. Maintaining good tests in a regression pack is important because regression packs will become increasingly prominent due to automation, Agile and DevOps

Back to Programme

    • Speaker


      Andrew Brown - Consultant, expleo, United Kingdom

      Dr Andrew Brown is a principal consultant at SQS. He leads an independent line of research into understanding why we humans make the mistakes that lead to software defects.

      He has 25 years’ experience in the software industry. Previous roles include Head of QA at HMV, Head of QA at a financial software house and a test manager in Japan.

      He holds a degree in Physics and Maths, an MBA from Warwick Business School and a doctorate from Imperial College.