Track Talk F9

The Rollercoaster Ride of Load Testing an AV Platform with 350K Users

Vimmi Walia

Yogesh Bawa

10:00-10:45 CEST Friday 10th June

Due to the COVID pandemic, the software industry is drastically transforming to service this new era of digitization by focusing on innovation and value driven agility. QA practices, test approaches and skillsets need to be aligned and upgraded so that we are ready for the newer challenges in testing. The client still needs delivery with quality, but now within strict budget constraints.

In this talk we will present a recent journey of carrying out performance testing on a complex use case of an audio video streaming platform. The test was to be executed with a load of 350K users within a limited budget. Hence, we had to opt for open-source tool, look for cost effective test infrastructure and develop a couple in-house utilities. The focus area for performance was not only overall performance of the system but also on monitoring the quality of audio video streaming when 350K users are live in the application joining different conferences running in parallel.

A varied technology stack was used like JMeter for load test, AWS for test infra to host load generator machines, Selenium with JMeter for simulating user actions like mute/unmute, share screen, turn on/off audio video streaming, Jenkins for creating a CI CD pipleline and to deploy the necessary components, trigger the test on all the created load agents, Terraform using AMIs for scale up and scale down test infrastructure, Influx DB to store the KPI’s related to audio video streaming as a time series data and grafana for report visualization. Along with that created a couple of in-house utilities for extracting the webrtc stats to present the streaming, utilities to push test data to different load agents and collect back the data and prepare a meaningful report from all the data from all the machines

This implementation was an experience packed with lots of challenges not limited to following:

  • Spawning and managing AWS instances from different regions.
  • Supplying data to all JMeter containers in parallel.
  • Storing stats from JMeter containers simultaneously in different S3 buckets.
  • Real Time monitoring for infrastructure along with data collation for results preparation.
  • Configuring S3 for collecting stats from JMeter containers simultaneously.
  • Building date-based folders in S3 buckets using terraform scripts.
  • Utility setup for WebRTC stats collation for data obtained from 300K users.


In this talk we will discuss these challenges and how we overcame them.