Still scared to ship to production on Friday afternoon? Even after reading our previous post on maturing your release process? Here’s some practical advice on evolving your releases — from a fearful event to a trivial byproduct of building software.

Building a release and go-to-market strategy is only possible when you have complete control over your releases and can perform them confidently. Today I want to look a bit deeper at the first big step which will help you release with confidence by mitigating risk. By following the guidelines in this post, it should be possible to start shipping fearlessly. Let’s dive in!

Start small

Big releases can be scary. When multiple people make changes to a large codebase over several days or even weeks, it’s impossible to keep track of everything that has changed. Not knowing or understanding what will be shipped is one of the main reasons why people are anxious to release software. The easiest way to make a breakthrough here is to drastically scope down what’s in a release. Are you scared to ship a new piece of copywriting? What about a new read endpoint? These are things that are small enough to ship safely and can help your team build confidence. You can gradually increase the size of your releases, but try to avoid shipping too much at once. It’s almost always better to do multiple releases of smaller changes.

Code review

A simple way to raise the confidence level of your team is to do more and better code reviews.  Code reviews not only help your team get a better understanding of what is being shipped to production, but they also catch potential issues that the author missed. As a developer, there are many things we can do to facilitate reviewing our code. Keeping the changes small and digestible is a good start so reviewers don’t get discouraged by the sheer number of changes they are expected to review. Proper descriptions and commit messages will help a reviewer understand the reasons why you are making certain changes and allows them to give you better feedback. Beware of pitfalls like having a single gatekeeper reviewing everything or having only senior members do code reviews. If you push things one step further, you can already introduce more eyes on every change by letting team members pair up and work together more. Pairing up brings the additional benefit of increased focus. It reduces parallel work and keeps your releases small and focused on one problem at a time.

One-click deploys

In order to ship smaller bits more frequently, it’s essential to remove as much overhead as possible when it comes to deployment. When a release takes multiple hours to prepare, roll out, and monitor there is simply no way you can do multiple releases per day. Automating your build and release process helps you speed it up, encapsulates the complexity, and simplifies the process from the outside. Ideally, every team member should be able to start a release with the click of a button or merging a change to a certain branch. When people can’t ship quickly, changes will eventually pile up and become a hard-to-manage risk.

Automated testing

The more complex a codebase becomes, the harder it is to keep releases in check. A small change can easily introduce issues in another — seemingly unconnected — part of the codebase. Refactoring towards decoupled components and adding automated tests to prevent regression issues will reduce the number of surprise bugs over time and help people get a better understanding of the full changes in each release. Having a growing suite of regression tests should increase stability and confidence, but it can slow down the speed of your CI pipeline. Sometimes it can even be beneficial to run your regression tests in your production environment to not let it slow down your releases. This way, you can detect issues in production before your customers do while keeping a fast pipeline to ship your next release.

Mean Time To Recovery (MTTR)

No matter how careful you are and how many tests you run, shipping often will not prevent you from shipping a bug. Even with diminished risk and small increments, something will end up going wrong in production. In those moments it’s essential to roll back to a stable release as quickly as possible. Rolling back to a previous release should be both faster and easier than rolling out a new release. As with any recovery process, make sure it works by testing it regularly. You don’t want to discover your rollback system is broken when trying to recover from a production issue.

Conclusion

In short, confidence in your releases will massively increase when you tackle smaller parts with more people. Working together puts more eyes on a problem and prevents work-in-progress from piling up. And just in case things go wrong, make sure it’s easy to rollback and redeploy. Good luck with your next Friday release!