Tutorial I

Testing the Test Tools

Michael Bolton

14:00-17:30 CEST Tuesday 28th Sept

“We shape our tools; thereafter they shape us.”

Vendors say that their tools make testing easy by making it easy to automate the GUI or API. Open-source developers wrap frameworks around other frameworks to make APIs and UIs more accessible to non-programmers. Managers hear the claims, see a demo, and want to invest time, money, or both in the tools. Then testers discover the differences between the sales pitches and the realities.

Everyone has the best of intentions; that is the starting point. The goal of this tutorial is to help people with strategies for looking at tools with clear eyes instead of rose-coloured glasses. We will examine testing tools, and develop models for evaluating them and how they might be used. Rather than simply touring their features, we’ll consider how tools might help us to test the product—and how they might not. That point of view allows us to see clear benefits and take appropriate cautions.

Our tools can lighten our load and help us to see the invisible, but when used indiscriminately, they can also become a burden and a distraction. Our managers and our teams need to be aware of this. We do too.

So we’ll examine the ways in which tools focus testers on specific tasks and on particular perspectives of the product. We’ll consider potential value from tool use that is often ignored. We’ll also consider subtle costs and hidden risks in strategies that centre on tools. We’ll look at how tools, while powerful and useful, can disengage us from our primary task: experiencing, exploring, and experimenting with the product to find problems that matter to our organizations and their clients.

Participants will be encouraged to share stories of their own experiences.