go back to the blog

Summary of Alan Page’s “Test Strategy Thoughts” webinar – by Shmuel Gershon

  • 14/10/2010
  • no comments
  • Posted by EuroSTAR

Webinar Conference, October/14/2010

Speed Notes! Summary of: Test Strategy Thoughts
As lectured by: Alan Page


I was inspired by the Agile Testing Days blog posts by Markus Gaertner. He posted – with minimal delay – complete summaries of the lectures he attended, which were written on the go during the many lectures.
This looked like a very hard task. And an intriguing one, because it can be very useful. Not only it is good for keeping a conference lecture in (cloud) memory, but also to share with others (on the web or back at work) the different classes.
Today Alan Page held a webinar about testing strategy, so I decided to try my luck at speed-notes on his talk. Below you see the result of my learning-by-imitating, please let know what you think and where I can improve.

Lecture Summary:

Who is Alan Page?
Alan Page is a senior tester at Microsoft working at the MS Lynk team, co-author of “How we test software at Microsoft” and of “Beautiful Testing“, and blogger at Angry Weasel.

The subject of the presentation was Testing Strategy. It was held over the web, using a Microsoft product called “Lync”. People like me who could not connect with Lync used phone and had to advance the slides manually. An interested twist in this webinar is that Alan invited all attendants to play with, test and try to break the Lync software and make the presentation go bad. How’s that for testers’ training? :)

Alan started by explaining that strategy is a shared vision about the roles and objectives of the testing team. Sometimes small teams can have a natural sense of this strategy, but larger teams will need something to get them converging on the same values. The final result of this document can be a poster, a PowerPoint slide, a checklist…

In his opinion, having this strategy clear (and documented) is important in order to have a better understanding of the testing process and the risks being taken.
For example, it can help explaining to our peers and managers about our testing job. It can help answering the dreaded “Why didn't you find this bug?” questions, as you can easily notice if it was a risk taken before hand when defining the tests. In one occasion Alan recalls, the strategy document had not been publicized, and when certain types of bugs weren’t found during the testing cycle this was a (bad) surprise for stakeholders involved.

What content enters the strategy plan?

According to Alan, the document should answer questions like:

• Who is the team? What do they know? What’s their experience?
• What they do? What should they do?
• What’s the current process? What I think should be? Differences between the two?

An important aspect of it is a description of the current testing situation (Where we are) and of the situation we aim to be (Where we want to go). Information about this gap will define improvements made to the testing process during the testing period.

Asked about who writes the document in his context, and how do other contribute, Alan answered that usually a Test Manager would author it. Or, on larger organizations, a senior individual contributor like a test lead or architect will take the responsibility.
And all the team contributes to it, either direct or indirectly. In his context, as he talks to the testers often, Alan is able to feel their opinion and get their feedback on the testing situation, which goes to the testing Strategy doc. This happens constantly during tests, too, as the strategy document changes as one goes.

So the strategy is a powerful tool in the hands of leaders, as it will guide the work and results of the team. Alan quoted Ron Heifetz on the book “ Leadership on the Line” who wrote: “Leadership is disappointing people at a level they can tolerate“, meaning that the strategy may not be pleasant, and it certainly would not be easy to attain.

Alan told the story that when joining the testing team, he noticed many things: There was good testing going on, but there also where overtly-focused testers that only knew about their specific part of the application. Additionally, there was no real exploratory testing.
One of the methods used to solve this was by holding brown bag meetings on exploratory testing. After a few successful sessions the benefits were noticeable.

When analyzing “going from here to there” or from a current situation to a desired one, Alan warns that the way is not a direct leap. It is not a direct line between the two points.

For example, if the current situation is such that there are many false positives in the test automation, you can’t just leap into “no false positives” mode.
You have to improve logging. Improve the oracles used. Improve the code. Improve the team running it. Etc. There are many pieces to be moved, not just one.

Back to the document format, Page recommends using one-pagers. Short documents including the things that are valued most.
The document should quickly convey “here's what we value the most” and “here's how we gonna do them“.

Once the strategy is settled, there is a need to follow up. Alan recommended 5 steps to it:

• Do
• Monitor
• Review
• Adjust

Show results

The results part is very important: People resist change naturally. Showing them results can help you combat resistance to change. Use numbers if you have to.
On this matter, Alan cited Dan Pink’s work on “A Whole New Mind“, where he shows that people are not, in fact, motivated by money. The number 1 motivator is progress.

After the talk was over, in the questions part, Alan explained that his document includes only the general approach to testing, and does not include detailed information on how the tests are done. In his opinion a second document (he calls it “Test Design” document) can hold this information.

My opinion from my note taking experiment

I ended up with a good coverage of the talk, but it wasn’t a perfect success: I did not have a finished-polished notes at the end of the lecture; I had to fill some blanks and fix dozens of typos.
Additionally, it is very difficult to think outside of the lecture when taking notes (like disagreeing or coming up with contrary examples). To type at speech speed, one (I) have to focus on what’s being said instead of what I think. It dangerously borders the point where it reverses the normal note taking benefit.
That’s why the text above is very little opinionated and impersonal.

Anyway, taking notes like this is good because you keep track of (and share) many details. Guess I’ll practice more until I’ll able to add my own opinions to the text.

What do you think? :) Let know in the comments below!

Blog post by

go back to the blog


Leave your blog link in the comments below.

EuroSTAR In Pictures

View image gallery