Bloggo back to the blog
Bridging the Difference Between Manual & Automation Testing-->
Everyone understands the need for automation testing for better (& faster) regression coverage allowing Manual Testers to focus more on functional testing and look for defects. In general testing teams are combination of manual and automation testers with manual testers greatly outnumbering the automation testers. There is also a more or less clear work division between these two different clans with a clear focus on tasks related to manual & automation testing respectively. However there are items regarding the difference between Manual & Automation Testing that can require attention.
One would think this work division to be quite normal given that the actual tasks involved in these two areas are quite different compared to each other. Let’s quickly analyze if there are any historical reasons for this being the situation. Traditionally test automation is considered a highly technical activity and hence falls under the domain of “developer” community. This is so as the average manual tester is more focused on the functional side of the AUT (Application under test) and less on the underlying technology aspect of the AUT.
This seemingly harmless fact has led to the test automation being a “prohibited” zone for manual testers and developers coming up with automation solutions which fall under the category of “of the developer, by the developer and for the developer”! Rarely the automation development is carried out by considering the manual tester community to be the end users of the automation solution.
With this kind of methodology, typical automations frameworks share one or more of the following (undesirable) characteristics.
• Difficult to understand (and use) by the manual testing team
• Test Case scripting being a purely coding activity even though framework design follows best practices like Keyword driven, data driven and modular design approach.
• Terminology used for variables, configuration properties being too geeky to be understood by the manual tester community
• Difficult to maintain (as perceived by the manual testers community anyway)
These characteristics play a role in alienating the manual tester community from the test automation. Let’s see what the side effects of this approach are:
– Increased cost of automation: Test Automation developed & maintained by the automation developers which are costlier resources than manual testers
– Teams working in silos: lesser synergies between the manual testers & automation developers
– Lack of specialization of roles: Automation developers need to understand a minimum level of business functionality to write effective automation scripts. Still a more desired scenario would be a manual tester well versed in business functionality also being able to write the automation scripts. Experience shows that intense technical focus tend to dilute the need to focus on functional aspects of the AUT – and vice versa
Let’s see if there is a better approach than the established one as discussed above. The better approach will be the one which makes the best use of the expertise of manual testers and automation developers and make their roles complement each other. To achieve this, we propose following way of distributing their tasks.
Desired Activities of Automation Developers
– Develop, design, maintain & enhance the automation framework as per the needs of the AUTs. To be able to do these activities, automation developers need to understand the spectrum of technologies involved, testing types, specific testing or validation requirements etc.
– Write stubs/emulators as needed by the testing phases in scope (or use third party stubs if that’s advisable)
Desired activities of Manual testers (apart from manual test execution)
– Automate individual test cases
Assuming they are able to automate, this would be the best option given their functional expertise
– Configure the automation framework to integrate it with specific test environments
Manual testers are better predisposed to take care of this activity than automation developers.
It is obvious that, such kind of work division best utilizes the expertise of both automation developers as well as manual testers.
Having said that, achieving this requires careful planning and a pre-mediated automation framework design approach with the result of incorporating below mentioned features
Make scripting test cases easy and eliminate need for coding knowhow
This can be achieved by using a better keyword driven approach in which manual testers are enabled to write the automation scripts using predefined keywords & passing them appropriate values (as arguments). The whole interface can be made easy to use by making it based on Spreadsheet or simple HTML. This eliminates need to actually write code (using a scripting/programming language).
Also it enables the testers to build new keywords using the existing keywords for easy enhancements of the automation framework
Make Automation more accessible (and less intimidating)
Provide an easy-to-use GUI which enables
– Easy script writing (better to complement this with a script recording feature)
– Configure the automation framework easily
– Manage automation runs (run/pause/stop/abort/schedule…)
– Monitor automation execution progress
– Troubleshoot in case of failures (better access & representation of test execution log)
Substitute geeky technical names/descriptions with more layman-like terms
– Use friendly, less geeky terms for system variables, parameters, configuration properties for easy understanding by the manual testers. E.g. it’s better to use “validate” instead of “assert”, or to use “parallel execution” rather than “multi-threaded approach”.
– The diagnostic messages appearing in the logs tend to be mystifying if written by the developers. It would be much easier to interpret the message “probably you entered a duplicate contract number” than a mysterious message like “ORA-00001: unique constraint (string.string) violated”. It’s more beneficial if automation framework allows the testers to customize the error messages as part of scripting or the configuration.
Make using automation fun!
Instead of making automation a standalone tool, make it integrate with other testing tools to increase overall operational efficiency
– Integration with CI tools for unattended regression runs every time a new build of AUT is deployed
– Integration with Test Management Tools to enable
o Triggering automation runs right from the UI of Test Management Tools
o Automatic execution results update in the Test Management tool enabling real time reporting in the dashboards of the Test Management Tools.
Advantages of the proposed approach
– Reduced automation costs – Manual testers (more abundant than automation developers in a testing project) writing automation test scripts rather than skilled automation developers
– Faster test automation – More resources are available to take up test automation. They are also better placed to automate test cases than automation developers due to their functional knowhow
– Better synergies (better resource utilization) – This is due to the fact that more logical task distribution between manual testers and automation developers to make them complement each other. Also with this approach manual testers get cross trained in basics of automation. And teams who cross train in each other’s skills tend to work well together.
The test automation industry is already moving in this direction with many tools vendors offering Scriptless automation tools incorporating desired features as mentioned in this article. Off course this trend is relatively new compared to the traditional automation tools like QTP, Selenium etc.
Before this trend becomes more mature and widely accepted, we need to come up with the automation frameworks designed by incorporating best practices suggested in this article.
Author: Dinesh Velhal
With the early part of his career spent in Development, Dinesh Velhal is currently focusing on developing new test automation offerings and leads the Open Source Testing Tools practice as part of the QEdge Testing Competency in Tech Mahindra.
With more than 7 years of experience in development, for the last 5+ years Dinesh worked on various management positions in large testing projects in Telecom Billing Domain.