Bloggo back to the blog
Product Review: StressTester Packs a Powerful Performance for HEB Manitoba-->
HEB Manitoba provides pensions and other benefits to eligible healthcare employees and their families throughout Manitoba through the Healthcare Employees’ Pension Plan and the Healthcare Employees’ Benefits Plan. With over 62,000 members, the Healthcare Employees’ Pension Plan is one of the largest pension funds in Manitoba and among the top 50 in Canada. (www.hebmanitoba.ca)
Norima Consulting, a San Diego and Winnipeg based IT consultancy. (www.norimaconsulting.com)
The Performance Testing Tool
StressTesterTM by Reflective Solutions. (www.stresstester.com)
A regulatory requirement made it necessary for the client to develop an online data gathering and verification portal for their members to be able to login, see their membership information and update it, within a government enforced window of three months. The testing team’s task was to conduct load and application performance tests of this ASP.NET MVC application prior to it going live in production. Periods of very high user activity were expected by the client, particularly as the government deadline for data input approached.
The performance goal for the application was to support 100 concurrent users through the portal application with the single-server deployment model and to find the breaking point. The simulated test scenarios involved up to ten transactions, which involved different web pages with variable data.
The performance testing strategy was to simulate application load through a scriptless web load and performance test tool and simulation program called StressTester. Load was simulated by running the test scenarios at moderate volumes, target volumes and break-point volumes. The testing team captured various metrics for each of these volume levels to demonstrate how the application would scale and perform in each.
The StressTester GUI provided easy to use and well-defined workspaces for all aspects of performance testing. The help links, information pages and in-tool video tutorials, which could be accessed at each step of StressTester’s configuration, and execution for that matter, were useful and offered significant help by ensuring that tests were correctly setup and realistic. This support was invaluable and intuitive and eliminated the need for any formal training on StressTester, which is a huge plus point.
We found that the most efficient feature of StressTester is the ‘User Journey’, which eliminates the need to write any test scripts. Instead, a wizard guided the team through the process of recording a User Journey and simulated various business transactions the team had outlined for the tests.
Three different User Journeys were recorded for Norima’s tests. StressTester then enabled the details of the User Journey to be configured, together with each of the step details within. The team was able to configure sleep time variations and pacing, HTTP specific configuration and specify validation criteria. We also set up parameterization to replace a recorded value dynamically from the session with the ones identified in our parameters. The inclusion of a dynamic data wizard here made this fairly easy to setup and configure automatically.
With each recording, the team verified that the tests were replayed correctly using the Quick Run feature provided in the ‘Run Components’ workspace. The setup of the Run Configurations was also done in this workspace, where we were able to specify the application to be tested, the User Journeys that will run on each injector, the ramp profile and the resource monitors to measure during the test.
Testing and Analyzing Results
Using an unlimited load, time-boxed StressTester license, the team was able to configure the actual performance test that would take place. StressTester provided, with a fine level granularity, the ability to set up things like the ramp profile that would put the load on the application and the user journeys to run in each location.
We added random delays between page requests to simulate the activity expected from real users. One test was run with 50 concurrent users. The test was structured so that the test would begin with zero users and add additional users until the target number was reached. In this case, it took the test ten minutes to reach 50 users.
Another test was run with 100 concurrent users. The test was structured so that the test would begin with zero users and add additional users until the target number was reached. In this case, it took the test 30 minutes to reach 100 users.
To find the breaking point, another test was run with 200 concurrent users. The test was structured so that the test would begin with zero users and add additional users until the target number was reached. In this case, it took the test 17 minutes to reach 200 users.
In all three instances, the team utilized StressTester’s ‘Analyze Results’ workspace to find out various information, such as which User Journey (specifically which web pages) were degraded the most, what were the slowest requests in these pages, what were the response times and how these could be improved.
This workspace included numerous graphs and charts displaying test run results such as average response times and number of requests made. It also displayed and reported any errors encountered during test execution. The team was able to drill down from each user journey to each step of the transaction and see the level of detail included there. All of this became very useful to document the findings and create the report used to communicate the team’s recommendations to the client.
The Norima testing team was able to correctly conduct the performance and load testing using StressTester and provide the client with analysis that showed how the application would scale and perform under expected load in real world scenarios once it was released. We confidentially concluded that the system maintains an acceptable performance level even with the maximum user load set according to the specifications.
StressTester was flexible and easy to use. There was no need to write any complicated test scripts and the GUI was well designed. The tool certainly ensured that the project went according to plan, met expectations and was delivered on time.
David has over 15 years of experience in the architecture, design, and development of enterprise-class, distributed web applications in the Financial Services industry. He has been engaged as Director of Software Architecture for the largest independent broker-dealer in the United States, where he provided valuable planning and budgeting guidance and ensured that projects and programs benefitted from standardized architecture and design best practices.
He is in the top tier of enterprise solution architects and is a proven unifying force between technology and business teams. He is able to conceptualise and communicate actionable solutions that deliver real business benefit. An exceptional leader with a tireless drive towards getting things done, he has a unique gift of being able to bring teams together to solve the most complex of issues.
Currently the Managing Consultant and CEO at Norima Consulting, David leads a balanced team of delivery-oriented consultants. Supported by a rich network of clients, peers, and vendors, David remains actively engaged in delivery, providing architecture consulting to financial services firms across North America.