Bloggo back to the blog
Testing in Agile-->
The Big Picture
Right from the beginning of a project, I think it is important to discuss the testing infrastructure within the project alongside the development one. If automation is appropriate, then during initial investigation the automation approach and the possible need for an automation framework should be on the list of things to look into (along with testing tools if you will be intending on automating the user interface tests).
I believe that agile methodology requires testers to think faster on their feet. There is not the same luxury of verifying and validating a well presented technical specification from which you would determine the planned approach, have the approach reviewed, and then take some time to stand back and let your brain process things a bit more before returning to it.
In Agile, you have to try and establish an understanding of a user story and its implications, often during the sprint planning itself. From a test perspective you then have to try and analyse the story by running through a mental list in your head of main use conditions, affecting factors, environment, edge conditions, basic tests and advanced tests.
Working this closely with development on a story can muddy the waters as to who is responsible for unit testing of the code, especially as test tasks other than unit tests do not always seem obvious when starting a story. I do not believe there is a hard and fast rule to this, but it is important to make sure it is clear to all those involved.
When planning you need to think about; the test infrastructure, any extensions you would need to do to it, unit tests, integration tests both low and high level, environment set-up, virtulisation if appropriate. Documenting tests before software is available can also help highlight scenarios that have not been considered in the current implementation, try to think through the most common ways a user will use a feature be it as well as the less obvious ways users can do things.
Try to make sure no story or derivative task is too big so you can actually address it within the sprint.
Tips for working in agile
The advantage of the iterative approach is that a testing influence is present at all times, to verify and validate each user story in the sprint as if each were part of a specification. Listen to the developers’ discussions and check that you have acceptance criteria that are achievable and cover all of the issues for the story. Check the tasks that are agreed on by the developers and make sure you understand the way they are working through them and thus the complementary testing tasks. Also do not forget to cover other tests that are ingrained in you as necessary depending on the software format. It is important to remember that for every feature of a story, the basic principles of testing still apply. Be sure to check that all estimates for a story include testing, reworking and retesting as well as the development.
Testing In Agile
In reality it is unlikely, if near on impossible, for all stories to be implemented and fully tested within a sprint. Often a feature is developed with some exploratory testing squeezed into the sprint and some simple issues addressed in the same iteration. It is still important to fully test each feature and apply the other tests even if that sprint has finished, they are tasks that need to be carried over into the next sprint. Some developers take up testing tasks towards the end of the sprint to help the team finish all the stories that they took on.
This is also where risk analysis comes into play; due to the increased speed in the turn around, it is essential to highlight the priority of the tests that you intend on running, and to appreciate the implications of not being able to run all of them.
At Red Gate, we try to apply as many testing techniques as we can to each user story within the sprint in an exploratory fashion, whilst also trying to automate as many of the test cases as possible. With this level of automation, we have continued confidence in the stability and functionality of the software on every build. We then define and document a minimum necessary set of regression tests for each feature, and then there is a final run through of these tests prior to release.
People may be wondering what is agile about the way we are testing; agile is itself based on the waterfall model, they are really just extremely small and quick waterfalls, one for each new feature. Working in this agile way allows us to theoretical be able to release the software at the end of each sprint. This is often the case if we are releasing something to our Early Access Program(EAP) however if it is a full release we occasionally have one or more stabilisation sprints prior to release.
Using agile in combination with an EAP allows us to really take advantage of the iterative approach and to create a rapid feedback loop with our customers as well. This allows us to get early feedback about the product or a specific feature, which in turn helps us to achieve what is really wanted by the customer at much lower cost than if we developed for eighteen months and merely released our interpretation of what the customer needs which would be much more likely using a non-agile approach.