Exploratory Testing
Although test automation is the biggest trend in software testing right now, only focusing your strategy on automation isn’t going to guarantee you a fool-proof QA process.
By diversifying your testing strategy with different methods, you’ll be able to cover more ground (i.e. untested code) and more unexpected discrepancies to your code and product.
Exploratory Testing should be a part of your testing strategy because it will test the effectiveness of your existing tests, discover code discrepancies, and alleviate bottlenecks where bugs hide the most.
In this post, QA Software Engineer, Pekka Pönkänen, tells us how he effectively performs Exploratory Testing and his advice on making the most of your sessions.
Why Exploratory Testing is important
In 2018 I walked the length of Japan, and one of my goals for the trip was to explore the country and its rich culture. During the trip, I decided to walk in different routes other than just following Google Maps, and as a result of that I had many exciting adventures. Undoubtedly, without having the curiosity to explore, I would have missed many unique places and interesting conversations with locals.
I firmly believe that in order to find new ways of seeing and experiencing the world, you need to be brave and curious about the unknown. The same goes for software testing. Always following the same paths, or tests, will get you expected results. On the contrary, if you go on an exploratory session of your application, you’ll be amazed to discover just how many bugs could be hiding or how much you can improve the functionality for the user.
If you want to try Exploratory Testing yourself, consider this your starting point.
What exactly is Exploratory Testing?
The term “Exploratory Testing” means that you are exploring the application and how it performs after different actions.
To truly understand the concept, it is essential to dig deeper into the roots of the terminology. The term “Exploratory Testing” was introduced in 1984 by Cem Kaner.
“Exploratory software testing is a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the value of her work…”
James Bach’s 2003 paper, “Exploratory Testing Explained”
Through exploratory testing, your goal is to find out how a specific area of an application is working while using your skillset to provide valuable feedback to your team. You want to find the nastiest bugs under the hood and make sure that nothing critical or dangerous can happen in the app.
Preparing for Exploratory Testing
Before starting the testing run itself, it’s good to have tools to write down ideas, bugs, and defects. Personally, I like classic pen and paper and bug-tracking software to write down thoughts and plans for the future. During the run, you will see application logs, automation ideas, new approaches for new testing runs, and bugs that need to be fixed. Having good software that captures images, videos, and notes can be very helpful in centralizing your findings and sharing your insights with your team.
Keep in mind that your notes do not need to be the most delicate piece of art. What’s important is that you can gather a small story around it for your teammates or stakeholders after the session. I have heard very successful use cases with testers doing mind mapping during exploratory testing. Having this visual aid in the process can be valuable to build off of your ideas and create themes.
Identify your goal
The first thing to get started in exploratory testing is to define what to test. It can be a known bottleneck, possible risk, new feature, or an area which has a lot of bugs.
As a software tester, you may know the places to look, and the development team can define the areas which need more attention. When planning the testing, remember not to make it broad so that you don’t lose focus. Here are a few examples to get you started:
What to do:
- Explore catalog page with a screen reader to verify page accessibility
- Explore login process with iOS gestures to verify that functionality is accessible
These tasks can be timeboxed and can be done in one session and short enough to be focused.
On the other hand, you don’t want to choose a task that has too many options or paths to follow. The best way is to keep it simple and focus on smaller tasks.
What not to do:
- Explore all possible mobile security issues to the system to discover any security-related threat
Once you’ve identified your goal and prepared your tools, here are the steps you can follow to complete an Exploratory Testing session:
1. Prep your session
After selecting an area to explore, design the test session. Once you know what to explore, you’ll get a flood of ideas about how to test different aspects — make sure to keep track of everything that comes to mind so you have a solid plan before you start testing. Write down your mission and prepare the notes about the way you want to proceed.
2. Setup the testing environment
Check that you have all credentials and access to enter the testing environment. Testing is pleasurable when you can focus on the testing itself, and when you don’t need to worry about usernames or unreachable servers.
3. Timebox and execute
Depending on the task, open the suitable logs, and monitor tools to log your actions during testing. Application logs are crucial to provide valuable information when things go wrong. While executing, keep your objective in mind, write notes, be systematic, collect info, gather ideas for the next sessions, and most importantly learn and explore your product.
Exploratory testing is more of a mindset than a framework
Exploratory testing is in fact a skill which is developing all the time while your skillset as a tester is evolving. The beauty of testing is that you never get bored of it because there is always room for improvement. Be curious about the application, try different approaches to execute testing, learn about your software, and share the knowledge.
At the end of the day, software development is a team sport!
Inform others about the state, risks, and any other concerns. Share your mind map as well as any evidence that you collected like videos, images and notes. You don’t have to have all of the answers, but sharing them with your team will also help others discover themes and insights which you wouldn’t have known otherwise.
——————————————————————————————–
Author: Pekka Pönkänen