Bloggo back to the blog
The Pleasure of Exploring, Developing and Testing-->
My parents tell stories of me when I was a small child and how I was a ‘terror’. I used to cause all sorts of trouble for my Mum and Dad. When I was aged 3, my Mum ran a Blackpool Boarding house. It was an exciting place to live. I loved playing with my Dad’s tools – he was a skilled handy-man and did all the ‘making and fixing’ jobs himself. One day I removed all the screws (that I could reach) from several downstairs doors and cupboards. My Mum was not pleased.
Dad left a sack of cement in a lean-to shed in the back yard. My Mum watched me from the kitchen window as I played all morning in the yard. I was purposefully marching to and from the lean-to with a teaspoon of cement. I deposited teaspoons of cement in the outside toilet until it was quite full. Of course, the cement set and cracked the bowl of the toilet.
Another time, I locked my mum and Dad in the kitchen. At 6.00pm. When they had a dining-room full of guests waiting for their dinner. Dad had to break the door down. I got a clip on the ear and he proceeded to serve dinner to the guests.
Another time I put powdered starch (used for ironing shirts) into my Mums’ custard powder – with predictable results. It was quite like Fawlty Towers, sometimes.
Then of course, occasionally (and every day in the summer), I used to run away. We lived about 200 metres from the Blackpool Pleasure Beach. The noise and excitement of the funfair was irresistible to a four year-old. After several hours of parent-panic, one of the engineers from the ride repair shop would call and let my parents know where I was. If it wasn’t the Pleasure Beach, I might have been spotted by a Policeman and the station Sergeant might call (since the Police station was just round the corner), “It’s all right – we’ve got him”.
I was quite a difficult child. You could say I had a vivid imagination, was a doughty explorer and adventurous. But I think you’d call it ADHD nowadays.
Because I was always ‘into things’ that I shouldn’t be, I was bought lots of books and games, but also construction sets and kits and working models to occupy me. Lego, Meccano, Scalextric, Airfix models, Hornby trains, balsawood gliders, were ongoing sources of entertainment right up to me leaving home and going to university.
I have to say, the urge to see how water, footballs, rocks, roller-coaster rides, starch, fire, explosives and electricity behaved and my parents’ rather extreme reactions to my experiments spurred me on. “The pleasure of finding things out” is how Richard Feynman put it. But overall, given all the things I broke, spoiled, lost or demolished, it was building stuff that gave the most pleasure.
I’m sure this is why I like working with software: you can build almost anything that you can think of.
In the last year or so, I’ve been exploring how to use devices that to date have been ‘dumb’, but soon may become part of the Internet of Everything. Some time ago, I bought two Raspberry Pi computers and some parts to build a robot. I had played with programming Python the Pi’s but hadn’t found the time to build my robot.
A couple of weeks ago I was browsing eBay and came across (and bought) an Arduino kit that packaged a dozen or so devices including a servo, a stepper motor, LED displays, two breadboards, a temperature sensor, a light sensor, patch leads and an array of other components. All for £30.
It was reminiscent of my childhood construction kits – lots of interesting bits to play with and assemble. On top of that, it requires a little bit of software to actually work – whatever ‘work’ means.
The kit came with a ton of PDF documentation describing all sorts of simple gadgets, starting with – a flashing LED. Connect your PC to the Arduino board with a USB cable. Run up the IDE on your PC to open the few lines of code required, download it to the board and watch it run. One of the LEDs on the board flashes. There is something very cool about writing software that controls hardware. I’m not sure I can explain why. It’s a most trivial thing. But seeing a physical device react to your code is really quite satisfying.
My First Attempt at Building a Robot
Last week’s wet Bank Holiday weekend, I decided to build the robot. It’s hardly an elegant solution. Then again, although the robot appears to be a clumsy effort, it does seem to work. The robot can be controlled from a browser on a device that is on the same wireless network. If I call it a prototype, then its appearance is kind of irrelevant, isn’t it?
Anyway, the hardware is basically a robot platform, two DC motors and wheels, a breadboard, an L294 motor driver chip, a Raspberry Pi with USB wireless dongle and two battery packs. The Raspberry Pi runs a one-page Apache website (built using Web2py web framework). This sends UDP datagram messages to the IP of the Pi. A service running on the Pi reads the messages and sends commands to the motor chip and DC motors. All the code is written in Python.
You can see a short movie of the robot here: https://vimeo.com/96324907
It works OK although the motors differ in their power a little so the robot moves in a curved path. (And perhaps the wooden floor is a bit slippy). A client running a little Python program could send messages directly to the service avoiding the need for a web server, but the web server allows any smartphone to drive the robot using a browser interface.
This is all very well, but what has it to do with testing?
The Fun Comes When you Test
Putting together little gadgets like this is great fun but the moment of fun comes only when you see the gadget work. This can only be when you test it. So the motivation for a gadget developer is to get to the point where the gadget can be tested as soon as possible, and also perhaps, to run tests that show the gadget ‘works’.
Now, our fear as testers is that our developers seek only the instant gratification of tests that pass and demonstrate the software does what is required – but no more than that. Yes, I’m *really* talking about software. More complex (and harder to build) features are paid less attention. More interesting and challenging tests (that are likely to fail) are avoided. This is a classic risk with, let’s say, inexperienced developers. They fail to test adequately – because they think that using a gadget is the same as testing it.
Exploration and Testing
I am currently working on a new description (another model) of testing which I will only outline here. You’ll have to wait a bit longer for a paper, but the essence of it is this:
1. We explore sources of knowledge to build test models
2. The models are used to challenge and validate the sources
3. The models are used to inform our (development and) testing.
The key to the model is that we separate the notion of exploration from testing. The purpose of exploration is to gather information from our sources of knowledge to derive models which we can use to challenge our sources and design or test our systems.
Sources of knowledge include documents (requirements, specifications, designs, standards, guidelines and so on), other systems, people (stakeholders, developers, users) and our own experience and understanding. The system under test is also a source and in some contexts, perhaps our only source beyond our experience. All sources are fallible.
Models can be documented, physical or mental. A model could be a diagram such as a flowchart, a state-transition diagram, a process-flow; a check list of design heuristics, a risk register or a decision-table, or it could be a sketch of a flow through system features or actions. Or it could be an entirely mental model. Our sources are fallible and so are we as humans. So our models are fallible too.
Our Brains are Designed to Build Models of Our World
Our brains are capable of modelling and remodelling our surroundings. These models allow our brain to send impulses to our muscles in highly complex patterns that allow us to move and reach our destinations without bumping into things. Our brain recalculates and recalibrates these models second by second. Modelling and visualisation are essential skills that play an important role in all of our everyday lives. The BEST robots on the planet are still quite crude in comparison.
You must have seen golfers practicing their swing before they take a shot. They rehearse and visualise the shot, the trajectory of the ball and the target. In many sports, coaches film athletes and talk them through their movements in great detail helping them to visualise so they can control their movement under great physical stress.
Athletes consciously model their world to achieve perfection or control. But models don’t only represent physical movement. For example, when Stephen Hawking lost physical capabilities, he invented a collection of powerful mental tools – models that allowed him to carry on working with his physics – without using a blackboard or written formulae.
We use this same modelling skill to develop and test systems. Our brains are incredibly sophisticated and fast modelling engines and mental modelling dominates our thinking.
Developer and Tester Modelling
I have hinted that perhaps, the modelling activity that testers do is the same as the modelling that developers do. Now, this is a quite speculative suggestion. Because I am a developer and because I test, I find that my mind constantly flip-flops between a melange of mental questions that refuse to go away: “What is required”, “How will it be used?”, “How shall I build it?”, “How will I know I’m done?”, “How shall I test it?”, “Is it finished, safe, ready?” These questions dominate my thinking whether I am writing code or testing it.
The questions that pester the developer and tester trigger the need to model the requirements, the solution and the tests. So let me suggest that perhaps, developer and tester exploration and modelling really are quite similar. Am I really suggesting that developers and testers explore in the same way? Well, yes, sort of – but it can’t be as simple as that, can it?
Developers explore their sources of knowledge to understand what to build and potentially, how to ‘grow’ their code into useful features. Whether the developer is using a test-driven approach or they adopt the ‘code a bit, test a bit’ approach, each test confirms that the incremental code the developer has just written matches their model.
Sometimes, just like a tester, the developer will try things out, not knowing what the outcome might be and then decide what to do.
The developer might take the outcome of a test and change or refine their model. Or they might ask for more information. Or they might fix some anomalous behaviour. At this time, the developer would not call these anomalies bugs because they occur as part of the learning process. As they model, they compare the software with that model. Through exploration and experimentation, the model and the software evolve towards an alignment that provides value to a stakeholder.
A tester has similar choices. The anomalous behaviour might expose a flaw in the requirement, the model or in the system. Judgement is required. The outcome might be more exploration, a refinement of the model, or a problem report.
Where is the Fun, Really?
It seems obvious that the fun for a developer is running tests and seeing that their software works. But where is the fun for the tester? Finding bugs? Breaking things? I think not – although there’s no doubt that all testers experience a little schadenfreude on occasion. It’s more subtle, I think.
The fun for both developers and testers comes when the system, as tested, behaves how the developer or tester believes it should. By that, I mean that the system appears to align with the developer’s and the tester’s MODELs.
It’s not that the system works or does not work that motivates. The pleasure is in having a model in one’s mind that appears to match the reality of the system and that alignment provides value to a stakeholder. We need developers and testers to recognise that modelling is critical and the fun comes where our models and systems are in alignment with our stakeholder goals.
Conference Chair 2014, Paul Gerrard
Paul Gerrard is a consultant, teacher, author, webmaster, developer, tester, conference speaker, rowing coach and a publisher. He has conducted consulting assignments in all aspects of software testing and quality assurance, specialising in test assurance. He has presented keynote talks and tutorials at testing conferences across Europe, the USA, Australia, South Africa and occasionally won awards for them.
Educated at the universities of Oxford and Imperial College London, in 2010, Paul won the EuroSTAR European Testing excellence Award. In 2012, with Susan Windsor, Paul recently co-authored “The Business Story Pocketbook”.
He is Principal of Gerrard Consulting Limited and is the host of the UK Test Management Forum and the UK Business Analysis Forum.