go back to the blog

Standards – a case for the Defence

  • 30/09/2014
  • no comments
  • Posted by EuroSTAR

Earlier this year, poll results on the EuroSTAR blog revealed intended use of the new International Standard for software testing, ISO 29119. The sample size was ‘about 60’ of self-choosing participants (i.e. small and not representative), but I was staggered at the figures, expecting many more in favour of the standard. The question “Will you be using ISO 29119 standards in your testing?” elicited replies as follows:

No, I will vigorously oppose such standards 38%
No, I will completely ignore them 19%
Yes, I think that testing can benefit from these new standards 19%
Other 24%

Now I am sure that the ‘other’ responses contained a variety of views, some in favour, some against, some unprintable and some frivolous (“I only follow standards produced by Brett Gonzales”). It may be wise to discount this category completely. Even so only a quarter of respondents are in favour of using the new standard. I didn’t know there were so many vehemently opposed to standards. Perhaps I will find out in the coming days, as the ‘no-standards’ police come knocking on the door!

For those that do not know me, let me make my position clear on three well-debated topics that tend to divide the testing community, though my stance on ‘standards’ is already clear.

Is testing a profession? (Yes)
Is testing certification a benefit? (Yes)
Are testing standards useful? (Yes)

I have contributed review-comments on both testing standards, and certification collateral, and also written (certification) examination questions. Were this time to be ultimately wasted, from my perspective it would be a shame, but I have almost no commercial tie-in to either of these topics. There are many others both in the UK and world-wide who have given much more time than I have. But for the sake of openness and honesty in the ‘standards debate’, I want to be clear. Neither do I approach the topic from the point of view of the ‘standards-police’ – oh yes, such people exist; there to crack down on those not using such-and-such a standard. Standards are only any use if they are ‘useful’ – a blindingly obvious logical tautology.

Almost all of us believe in some ‘standards’; it is how society functions. Even the mandatory standards in life have to be filtered using common sense. Road usage for motorists in the UK is governed by The Highway Code. Some parts are backed up by law (priority at road junctions, speed limits), others are advisory (informative stopping distances at various speeds in differing traffic conditions). In the UK, we drive on the left side of the road (a mandatory part of this standard). If you only ever drive on the left, you may have a problem when trying to pass a flock of sheep, circumnavigate a landslide, or when traffic-controlled road-works are in operation. Only yesterday, I witnessed a motorist legally pass through red traffic lights – because it was very congested and a siren-sounding, lights flashing ambulance was trying to reach the hospital. Servile observance of the mandatory law could have caused a loss of life in the extreme case.

So why do I advocate testing standards, in the way that we would all understand the words? Three quick points: they work for me, I don’t want to go back to the free-for-all practices of yester-year, and standards can school the untutored.

They Work for me

I have used some form of testing standards for a number of years, as a hands-on, at-the-coal-face tester. I listen to others, take advice and read reasonably widely, so ‘Standards’ are not my only place of reference and source of inspiration. My briefcase is a walking tool-bag, and often has actual (e.g.) Test Strategy documents that I can draw upon, used by myself or others. Sometimes, the project at a new location is nothing like others for which I have documents. Then I am faced with having to create a Test Strategy from scratch. In such situations, turning to the appropriate part of ISO 29119 may be beneficial. There are others who when faced with writing a Test Plan, think of IEEE 829, and recall a well-known acronym. “Now, what does SPACE DIRT refer to as headings in a Test Plan ……?”

Another situation where worked examples are a real boon is test techniques. I have quite a number of books on software testing. They are not all used to prop-up the bed with a missing leg, but it can be difficult to remember where a good worked example of Equivalence Partitioning as a technique can be found, to share with those (both within testing and outside) not very familiar with the detail. I used to turn to BS 7925-2, but now go to the under-draft ISO 29119 Part 4. This is dull, boring and all too familiar – until you need it in a hurry. Then it is invaluable.

No return to the free-for-all

It can’t only be me that remembers the bad old days, the time when there were little common understanding because we all spoke a different language. The same activities were being carried out, but they were called not only different terms in different companies, but even different names within the same company. We were saying the same thing (i.e. the same ideas) but did not know it because the words were different. One value for ‘Standards’ is to avoid returning to this type of situation. “Hold-on, Peter”, I hear you say, “That could never happen. Things are different now. We could never return to those days. There is the Web, Twitter, and a whole host of ways that information is now accessed”. Precisely. Masses of information, but where to go for guidance in the quest for knowledge of how to proceed? There is almost too much.

I am not advocating that a standard (any standard) would tell you what to do in your situation. It is your situation, not that of any standards organisation. It can, however, frequently be a pointer to you, and even a source of where to get more information. If you remember an earlier point, standards need to be aligned to common sense. Otherwise, it is just the bland leading the blind.

A Standard has some mark of acceptance, to differentiate this information Ifrom the myriad of other sources that are available. It has some industry ‘seal of approval’ – this may be very helpful, written both in bold, and with the reality check of common sense needed.

Standards can school the untutored

I recently attended EuroSTAR 2013 in Sweden – what my wife called ‘The Gathering of Geeks in Gothenburg’. Over my eleven visits to this conference, there have been perhaps 7,700 delegates in total, but probably only half this number of different testers (my guestimates for both figures). How many testers are there in Europe? A lot more than 3,500! Until recently, there have not been those entering the profession by choice – most fell into testing and a good number were trying, in some cases desperately, to fall out again. Even if individuals consciously enter into testing as a career, there are many more testers than attend conferences, contribute to testing debates on Twitter or read testing books.

Don’t get me wrong. We as testers need to continue debating what works well, what is great in particular circumstances, how to achieve better test coverage and the like. It is sadly true that this may not be at the top of the agenda for lots of those who test. Many of those who engage in testing have not read a single testing book, and it is people like me that raise the average number of testing books read by those engaged in testing to be almost one per tester – according to a keynote speaker at EuroSTAR in 2007. The situation may have changed a lot in 6 years, but I am not seeing the evidence.

So if individuals don’t go to testing conferences or read testing books, where do they get their information from? It could possibly be from a certification course, or studying for a qualification. You may not rate the value of such schemes (a topic for a further conversation, surely), but now over 300,000 have obtained the ISTQB Foundation certificate worldwide. At least in the syllabus, standards are mentioned, and some key syllabus themes are very much based on existing standards (IEEE 829, IEEE 1028, IEEE 1044 and BS 7925 parts 1 and 2). Standards are being talked about, and can be a solid foundation for those that don’t attend ‘The Gathering of Geeks in Gothenburg’.


A lot has been written about Standards in the testing world, both for and against. It can be an emotive subject, but there are valid points on both sides of the argument. The ‘standards camp’, if I can use that term, can certainly take on board some of the good ideas espoused by those opposed to standards. There may be no end to this particular debate in the foreseeable future. Let’s keep talking on this; as we do this, we can begin to achieve what we all desire – the advancing of the profession, with more and better testing.

My one hope is that “The standard of no-standards does not itself become a Standard”. Then its protagonists would wonder how on earth that happened.

Blog post by

go back to the blog


Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery