Continuous delivery and continuous deployment offer testers opportunities for growth

Development practices are constantly changing and as testers, we need to embrace change. One of the changes that we can experience is the move from monthly or quarterly releases to continuous delivery or continuous deployment. This move to continuous delivery or deployment offers testers the chance to learn new skills.

A project that makes monthly or quarterly releases has a familiar rhythm and the team builds towards the release date. The testers need to test all the cards and carry out manual regression testing. Every test in the manual scripts needs to be executed, and possibly a report needs to be made on the results of these tests. Once the release has been made it is possible that there will be bugs that were in the release that need to be fixed. The testers will also need to start running the same manual regression tests on the next release and report on them again. Testing for monthly or quarterly releases is a repetitive process. This process has been compared to the Greek myth of Sisyphus who had to roll a stone to the top of a hill, and then when the stone rolled to the bottom of the hill he had to roll it to the top of the hill again.

Continuous delivery can be defined as “when all developers are working in small batches on trunk, ….and when trunk is kept in a releasable state, and when we can release at the push of a button” [1]. A team that I worked with moved from making monthly releases to continuous delivery. The team kept the main branch in a state in which it could be deployed if needed and the team made weekly releases. Continuous deployment can be defined as when, in addition to the practices that support continuous delivery “we are deploying good builds into production on a regular basis through self-service (being deployed by Dev or by Ops) [1]. A team practicing continuous deployment deploys to production each time code is merged to the main branch.

Teams that practice continuous delivery or continuous deployment use small batch sizes. That means that the “batch” of code that is deployed is small. “The theoretical lower limit for batch size is single piece flow, where each unit is performed one at a time” [1], this is what happens in continuous deployment where each merge to the main branch is deployed to production.

Teams that practice continuous delivery and continuous deployment try to create a flow of work at a sustainable pace and so should “enable and inject learning into daily work”[1]. Teams that make monthly releases build towards the release and so can not create this constant flow of work at a sustainable pace.

When a team moves from monthly releases to continuous delivery or continuous deployment a change that occurs is that there is no release candidate. Testing is not done on a release candidate instead, it is done on feature branches that have been taken from the main branch and when they are merged back into the main branch they must be ready for release to production. The main branch is kept in a state where it can be released to production. For the tester, this means that the testing on a feature branch has a different pattern than that on a release candidate.

When testing on a feature branch, you need to be confident that the new feature or fix in the feature branches does what it is supposed to do and that it has not caused any regression. When testing a monthly release you can have time to execute manual regression tests, but if the main branch is to be kept in a state that it can be deployed to production this is not possible. Regression testing needs to be automated if you are using continuous delivery or continuous deployment.

Regression testing on monthly releases often includes running a large number of manual tests; however if your team is using continuous delivery or continuous deployment regression testing is usually automated via continuous integration. Continuous integration (CI) “means that every single time somebody commits any change”[2] to the code the change is integrated into the codebase. This entails running automated tests prior and after the code is merged into the main branch. This provides an opportunity for the tester to learn how to understand CI. The tester needs to understand CI, including what tests run as part of CI. There will always be gaps in the tests that run on CI. If the tester knows what the gaps in CI are they can work out how to automate tests to fill them and execute manual tests to cover the gaps when this is required. Making mind maps of the tests that are automated can help identify gaps in the automated tests.

Testers can also get involved in automating regression tests themselves and in this way testers can help to prevent bugs rather than find them. There are great free resources such as Test Automation University, LambdaTest Certifications and Exercism which can help testers gain the skills they require to automate tests. There are also lots of resources to learn how to use javascript to aid testing.

That regression testing is automated creates time for the tester that they can spend on exploratory testing. Exploratory testing is a powerful way to uncover issues, and so will help the projects that the tester is working on. Having extra time to do exploratory testing will also help the tester develop their exploratory testing skills.

Projects that use continuous delivery and continuous deployment also tend to have a microservices architecture. Microservices are services which have independent testing and deployment, and each service is simple[1]. The tester has the opportunity to learn about the microservices, and ways to do this include talking to the developers, studying any architecture diagrams that exist, reading the readme files for each service in GitHub and attending the developers’ meetings. [4]

Testers can build their relationships with developers by helping them test their code. The tester can share their knowledge of testing techniques, such as boundary value analysis, with developers as this will help them test and produce better quality software.

Release processes for monthly releases can involve a certain amount of pain for testers. Sometimes we are asked to take the responsibility of making the release decision even though the tester is often a junior member of the team, other times testers have to take part in large committee meetings of stakeholders that decide if the software can be released. Release processes for continuous deployment and continuous delivery should be automated this means that making releases does not put pressure on the tester. The tester’s input to the release in continuous delivery and continuous deployment is their testing, this means that we can focus on our testing and learn new testing skills.

Development teams are not autonomous, they are open systems whose work affects other teams and is affected by other teams. This is systems thinking and systems thinking contributed to continuous delivery and continuous deployment. Testers can learn to use systems thinking to enhance their testing and support their team. This can help the tester think beyond their role to understand what other systems are affected by their team’s work and what systems affect their team’s work.

One of the lessons of systems thinking is that everyone shares responsibility, so no one person should be blamed when something goes wrong. This view should also be at the heart of every agile and lean development team that implements continuous delivery or continuous deployment. This is something that testers should learn from and take to heart. When there is a failure we need to learn from it and not blame someone.

Teams that make monthly releases can find that after each release there is a flurry of activity fixing regression bugs that were deployed in the release. This does not happen with teams that practice continuous delivery or continuous deployment. Releases will be deployed multiple times a day using continuous deployment, and software will be deployed regularly with continuous delivery. These regular releases give the development team an opportunity to recover from bugs and incidents quickly as fixes can be deployed quickly. Once a fix has been deployed the tester can offer to take the lead in doing root cause analysis to find the root cause of the bug or incident. The tester can learn to use the Five Whys and Ishikawa Diagrams to do root cause analysis.

Testers can also support the work of their teams by producing metrics that help the team measure their improvement of quality. DORA metrics[3] were identified by the Accelerate State of DevOps survey report by DORA. These metrics are designed to help teams that practice continuous delivery and continuous deployment find areas for improvement and know how they are doing. These are different to the metrics for monthly releases as they are not about how many bugs have been put into production but are about how quickly a team recovers from an error.

Continuous delivery and continuous deployment offer testers opportunities to grow and learn new skills so testers should embrace the opportunities when their teams move to continuous delivery and continuous deployment.

References
[1] The DevOps Handbook by Gene Kim, Jez Humble, Patrick Debois, and John Willis
[2] Continuous Delivery by Jez Humble and Dave Farley
[3] Measuring to improve quality by Janet Gregory and Lisa Crispin
[4] How Can I Test This? by Nicola Lindgren, Mike Harris, Phil Wong, Suman Bala, and Shawn Shaligram

Further Reading:

• Accelerate by Jez Humble, Nicola Forsgren and Gene Kim

Mike Harris

https://testandanalysis.home.blog/

Mike has been working in testing for 20 years and is the tester at Geckoboard. He has been a member of a test team, a Solo Tester and a Test Lead. He has also worked as a part of waterfall, lean and agile teams. He has set up and led a Testing Community of Practice and been part of a successful agile transition. He is also A Chartered Fellow of the British Computer Society and is Vice Chair of the British Computer Society’s Specialist Interest Group in Software Testing. He also contributed to the e-books Testing Stories and How Can I Test This?. He has also had articles published by the Ministry of Testing and The QA Lead. Mike has a B.Sc.(HONS) from Middlesex University and is an Associate of the University of Hertfordshire. Mike has a blog at https://testandanalysis.home.blog/ you can follow him on mastodon: https://mastodonapp.uk/@TestAndAnalysis

See author's profile