Bloggo back to the blog
from the webinar: Introduction to Test Strategy-->
Below are Rikard’s responses to questions posed at the ‘Best of EuroSTAR Conference 2012’ webinar entitled: ‘Introduction to Test Strategy’ from Monday, 3rd December 2012. If you wish to download a copy of Rikard’s presentation slides, please follow this link.
Thanks to EuroSTAR for letting me have the webinar, and thanks to all who listened and asked questions.
It is freely available at http://youtu.be/OZiE9eApOXY
Here is partly new answers to questions from the audience.
How is the test strategy different for testing embedded systems?
I should have said at the beginning of the webinar that these thoughts might be useful in any type of project. There are always details that you need to know about, for embedded systems you might need to consider temperature for instance. But you still want to find out your mission, you need understand the context, you want to know about which quality characteristics that matter, and you still want a diversified strategy, that is specific and justified for the unique situation.
How would the Test Strategy be applied to an Agile TDD project?
My hope is that the ideas can be useful in any type of project, but it is you that have to think and decide how. TDD is firstly, according to me, a development method, that will be just one of many factors that can influence the test effort. If we focus on the short iterations part of Agile, I would create a unique test strategy for each sprint. If possible, I would deliver it verbally during a planning meeting, and hope to get feedback from the other team members.
What is the SDLC stage of test strategy (start and end)?
The living test strategy, the ideas you have in your head, is ongoing, and stops at the same time as the project ends. The documented test strategy is formulated when it brings value to the project.
How you do view resource negotiation for the project?
I believe resource negotiation gets better when you can discuss what testing can accomplish. So I think a test strategy can help here, maybe you write a strategy for the resources allocated, and can talk about the testing you would like to do, but probably won’t have time for. After that it is a (well-informed) risk decision, and testers do the best they can with what they get.
Regarding test strategy for a product that is built and delivered in monthly increments: How would you address these types of things in a team that works with same product but different things coming into the product continuously?
There will always be a test strategy, and since it is such an important thing (communicates with mission givers, guides testers) it is worth doing well.
I would probably opt for writing a new strategy each month, and probably change the methods along the way (a way of testing is often best the first time it is used.)
Maybe a high-level test strategy (test policy?) could help.
Which comes first, test plan or test strategy?
I see the strategy as a part of the test plan, and I think it is the most important part. Without a strategy, there can’t be a justified plan. However, I know there are a lot of test plans, where the approach part is a Barnum strategy meaning nothing at all, and maybe even copied from a different project. I hope this is addressed by good testers and test leads making the right decisions along the way.
What strategy should one adapt in an environment where production issues are recurring in previously released modules/features?
Difficult to answer without knowing details, for instance I don’t know if the “production issues” is a real problem. Maybe only a few issues need to be addressed, and the customers are very happy with quick support.
I don’t know if this is a testing problem, a coding problem, a resource/project/product management problem, or an infrastructure problem.
But let’s assume it is a testing problem, and in that case I would advocate testing in different ways; I would encourage creativity and trying to find new ways. Even though most production issues are unique, I hope we could analyse the previous important ones, imagine how we could find these issues with testing, and then generalize one step further to find test strategies we believe in.
How do we get people to tell us the mission? If you ask why, they cannot answer. If you give examples, they want it all or just select some.
(How to find out what’s important if you are not in direct contact with all stakeholders?)
You can try to ask them what they are afraid of, that’s a different kind of question that can lead to very good answers. You can ask them if you should experiment with no testing for a while.
If you are unsatisfied with answers, I would try asking someone else. I might look at other sources as well, if “find important problems” is a probable mission, the marketing material gives some answers.
Does test strategy need to be written? For who do you write it?
There is always an unwritten test strategy. It should be written if it is needed…
I usually write it firstly for stakeholders, and secondly for testers (I have a lot more chances of discussing with them later.)
Do you think deciding the way testing will be managed is part of test strategy? E.g. is it test case based, session-based, or something else?
I think you can put whatever is important and appropriate in your test strategy.
So if it is important, and it isn’t best trusted to testers, managing testing can be part of the test strategy.
Where there are multiple partners responsible for delivering a complete solution, how would you capture the shared responsibilities/test phases? How would the test plans help to manage the testing in this instance?
I would communicate a lot, generating common understanding. I would discuss “healthy test overlapping”, I would encourage a lot of people reading test strategies (that shouldn’t be too long.) I assume the distribution of responsibilities will be covered elsewhere.
How do I write a Test strategy if the functional requirements are not clear /or not detailed?
I don’t think functional requirements are needed to write a test strategy.
They might be useful to sample and understand how they are written.
Unclear or non-detailed functional requirements could be a bigger problem for developers than for testers (we test with what we have.)
How will test design help in test strategy?
For me, test design is an open word, that can mean deciding exactly how to test something, but also just to come up with a vague idea about something to test. It can be done in advance, or on-the-fly when testing.
I see test strategy as a part of test design, and test design as a part of test strategy.
Test strategy can be seen as about “big” test ideas, and test design is about “small” test ideas.
They help each other, a lot, and in both directions.
Some (informal) test design is probably necessary in order to create a specific test strategy; and some (informal) test strategy is probably necessary in order to design tests.
I hope you do a lot of both.
Feel free to ask new or follow-up questions here!