Track Talk F3

How Wayfair Tests Software

Evgeny Tkachenko

09:00-09:45 CEST Friday 10th June

The online retailing market is very competitive and demanding, so you need to deliver new features/ functionalities to your customers as quickly as possible. In these realities, every wasted minute can cost you thousands of dollars.

We release our code to production hundreds of times a day with confidence in the quality of the products. We have 6 Quality Engineers (QE) and 200+ developers in Catalog Engineering. We educate on, report on, create, execute, and monitor automated tests enabling engineers to ship features with efficiency and confidence. We advise teams on their approach to quality with design, test-automation environment-setup, reporting and maintenance, architecture, code-analysis tools, and requirements-analysis. This results in promoting software quality by design.

The key to success is in the shift left (scrum teams start from the beginning):

  • analyze requirements
  • use static сode analysis, vulnerability scanners, and code reviews
  • write unit tests
  • automate only what matters most
  • provide training and coaching on what, why and how to test/automate
  • visualize the health of the product and test automation
  • enable continuous delivery through deployment pipelines
  • use incremental rollout for “big” features
  • constantly monitor our applications in production
  • If something goes wrong we rollback the changes

 

We did not transition to our current team structure (without embedded Quality Engineers within the scrum teams) in one day. First, we had to get our team members test-infected (in a good way), inspired by good role models from adjacent teams, and engaged in the test automation process. Only then could we cultivate the culture in which the whole team is responsible for the test automation and quality in general. Now we empower autonomous teams across Catalog Engineering to deliver value rapidly, repeatedly, and reliably through building tools for testing, defining standards/best practices, and training teams on test automation.