Track Talk Th12

How a Confidence Monitor Supports Decision Making in your Organisation

Ard Kramer

15:30-16:15 CEST Thursday 9th June

Testing is about sharing information! Information that will give confidence in your IT solution. What information could we use to support the confidence in a system that is vital to running trains in the Netherlands? (If the system wasn’t working properly, the public rail transport really would come to a standstill – a nightmare!). This was our challenge.

We created a confidence monitor that shows the level of confidence based on measuring a wide diversity of indicators in one graph. Over time the trend in confidence helped our stakeholders in their go/no go decisions. The ‘we’ in this experience was the test manager of the main project team, and the quality supervisor of the steering committee. We started with determining how to inform the various stakeholders in such a way that they could easily grasp the essence of the information – and get enough information to establish the confidence needed at the moment they decide to go live.

For our “confidence monitor” we selected a range of about ten different indicators tuned to the needs of our stakeholders. How can you apply a confidence monitor in your organisation?

Based on our experiences, these are examples of important lessons for informing your stakeholders:

  • Provide the right information on the right level in your organization. So, make sure there are different levels of information. We used smileys for high-level info, graphs for overviews, and text for details.
  • What kind of information supports the level of confidence? Don’t just use indicators that contain hard evidence: e.g. uptime of the new system. You should also use soft evidence: measure the level of confidence that end-users or operations managers have.


In my talk, I will give many more examples of relevant indicators. With a confidence monitor, you can provide trust and peace of mind, instead of a test report being the traditional bearer of bad news about problems in a system.