Bloggo back to the blog
Quality @ Speed – Transitioning to Continuous Integration and Continuous Delivery-->
A lot of companies want to put more speed in their delivery of software to customers and with that the same or preferably a higher level of quality. Continuous Integration and Continuous Delivery (CI/CD) is one way of achieving this. I am involved in such a transition at the moment. Below are a few of the learnings I had so far.
For those who are not as familiar with the above terms, here are some definitions:
- Continuous Integration: the practice of merging all developer working copies to shared mainline (also called master or trunk) several times a day.
- Continuous Delivery: a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time.
- And some companies even extend this with Continuous Deployment: this is the next step of continuous delivery. Every change that passes the automated tests is deployed to production automatically.
To be able to implement CI/CD a company has to invest in 5 CI/CD principles.
- Everyone is responsible
- Integrate early and often
- Automate repetitive tasks
- Keep increments small
- Continuous feedback
From a testing perspective we can contribute to all these principles. Below a few examples per principle.
Everyone is responsible
In an Agile team everyone should be responsible for all tasks that have to be executed. So also testing. A good way to achieve this is to have the entire team think about the Test Strategy for a specific sprint: what tests have to executed on which environments and with what depth. This will help the team in planning for a sprint: some user stories are easier to test than others. Another good practice to help this principle is to use Exploratory Testing for the manual tests. Not only “testers” will be involved in this, but also the other roles in a team. At the end everyone is also responsible for quality.
Integrate early and often
Continuous Integration is all about integrating as often as possible. Preferably multiple times a day every developer integrates their code with the mainline of code and so with everyone else’s code. This means that also tests have to be executed often. A lot of test effort will go into Unit Testing, so maybe more and earlier than what people in the old world were used to. Here also testers can help: what about Unit Testing as a Team. And parallel to this the team has to decide what other tests are necessary: regression testing, E2E testing or whatever tests are needed to finish a sprint successfully. The success of the integration also has to be tested continuously.
Automate repetitive tasks
To be able to implement CI/CD, as much as possible of the activities in a delivery pipeline should be automated. This will add to the speed and quality in a pipeline. Start with all repetitive tasks for example from committing code, to quality checks and to automated testing. And here is where it sometimes get tricky for testing. I hear a lot: all testing should be automated and that is something I don’t believe in. Yes, we can automate all our regression testing and whatever checks we do, but manual testing will be needed for tests where human interpretation is required. And here is a good case for Exploratory Testing.
Keep increments small
To keep speed and quality in a pipeline, the increments that developers are working on should be kept really small, maybe just a few lines of code. When something goes wrong in the pipeline it is easy to solve it as it is clear which small part of the code caused the issue. This can be taken out of the pipeline, it will be fixed and committed again. So also from a testing perspective we have to think of the set-up of our test cases. Here also version management gets important. We have to know which versions of our test ware were used to test which version of the software. Aligning with the version management system that is used in a pipeline is key. And this can also be automated.
With an automated pipeline a team automatically gets continuous feedback. This will differ from a build that is OK and implemented in Production to issues happening somewhere in the pipeline. Especially in the beginning of setting up a pipeline, a team will get a lot of feedback information, so a team should be prepared for this. And be ready to use the feedback. Is a regression test constantly failing, what is the reason: is the test wrong, do we use the correct test data or could more unit testing be executed or… There should be time in your sprint to be able to interpret the feedback.
CI/CD makes us think about testing again and how we can plan and execute it best in a sprint. This is challenging but also a new addition to our testing profession. We have to work together with all roles in a scrum team. And yes, we do have to get more knowledge of other professions which makes us more T-shaped. Of course things are frustrating sometimes when starting of this process as not everything works perfectly from the start. But at the end I think it is a lot of fun as well. And at the end it gives a lot of fulfilment when more and more of the repetitive (and so) boring stuff is automated which gives you time to think about and execute the more challenging tests.