Releasing a product or service is hardly straightforward. Even a brilliant team, good test coverage, solid continuous integration and a robust release pipeline cannot save you from the ‘release rush’ ( rush-like feelings and actions when closing in on the finish line for a product release). This happened to me quite often. Somehow releases always turned out to be more stressful than they needed to be. For instance, teams would start early and often work without break until midnight.
Over time, to prevent this, we studied the system and devised a plan to improve our release process. This blog documents my learnings and is meant to be a practitioner’s guide that can help avoid the release rush.
Pre-release checks
Before a release, we’d recommend listing all the knowns, the unknowns and performing the following checks:
Completion of the accepted features
Acceptance testing
Sign-off from the product owner and agreement on the release date
Approval from other stakeholders such as security, licensing, IT, etc.
Feature completion of upstream or downstream systems
Feature toggles to be enabled/disabled
Bug bash
Regression testing — manual or automated
Hotfix plan, if something goes wrong
NFR/CFR checks like performance, browser and platform compatibility, etc.
Appstore/Play Store publishing
Creation of a new environment
Backend and frontend release
Data upload
Third party licenses, renewals, contracts
Documentation — release notes, manual, known issues, etc
This list is not exhaustive but is a good starting point. For your project, you might need a different list or you might want to evolve this to make it foolproof over time.
Post-production checks
In one of my projects, we thought the release was successful. Only to later realize that we had accidentally uploaded a previous version of the data. Such mistakes can also be avoided by following a thorough checklist. Some essential items to check are:
Core functionality, especially if the features are entirely new
Performance across different supported devices
Data integrity, if it involves SaaS or a major migration
Logs, in case of integrations with new upstream or downstream systems
Performance dashboards, if you are expecting heavy loads (like a booking system)
Other monitoring dashboards or Sonarqube
Automation
Automation is good, so long as you know exactly where to put the machine.
To ensure consistency and reduce dependencies, automate anything that can be automated — whether it’s a complex process or a mundane task. Here are a few things you can automate:
Build and pipeline
Deployment process
Testing — regression, sanity, performance, etc
Calendar reminders for renewals
Data upload
App publishing
Reminders for document publishing
Post-production sanity
Sonarqube
Release template
Document the checklists in a place that is accessible to everyone involved. Ideally, create a template and use it each time a release is scheduled. Documenting is a breeze in confluence; you can use wiki too. You can also group the list based on teams, tools, or processes for ease of use.
Backtracking dates
These days it is common for teams to schedule release dates even before deciding what is being released. Use the release template to create a timeline by which each task needs to be completed.
Project accountability
Set up a meeting with all the stakeholders involved with the release of the product and ensure:
Clear list of teams/people involved and corresponding dependencies
Understanding of timelines and events
Sharing of suggestions/objections/changes etc.
Discussion around possible RAIDs
Then, invite people to sign up for tasks on a copy of the original document/template. For instance, a product owner needs to know when to give the go-ahead. The IT team needs to know when to renew a license, create new servers/environments, etc. The security team needs to plan its final audit. And so on.
By the end of the meeting, you need two things: A date (of completion) and a person accountable. The person accountable can, of course, seek help from others. But it is essential that an individual is accountable to ensure the task gets done by the said date.
Following the plan
Set up the cadence for a periodic check-in on the plan. A meeting for every iteration is a good start, but keep a daily tab when you are in the last iteration of the release. The initial few meetings might be longer, but after a few iterations, it becomes quick. Even async communication modes like chat/emails could work.
Use these periodic tabs to solidify the plan. Recalibrate dates/tasks as you need. Make this process as organic as possible.
Release 15, September 2021
|
||||
Sl.no | Task name | Assigned to | Date | Completed |
Code releases
|
||||
1 | Front end (web) and back end release | Jaya (QA, Dev team) | 14 September | |
2 | Andriod app release | SPD (QA, Dev team) | 10 September | Done |
Testing
|
||||
3 | Android app regression | SPD (QA, Dev team) | 9 September | Done |
4 | Front end and back end regression | Jaya (QA, Dev team) | 12 September | Delayed. ETA 13th. |
Other dependent teams
|
||||
5 | Final security audit | Simon (Security team lead) | 8 September | Done |
6 | Upstream service release | Rajesh (QA, Upstream team) | 5 September | Done |
Post release checks
|
||||
7 | Post release sanity | Sneha (BA, Dev team) | 14 September | |
8 | Monitor Dashboard | Shakier (Dev, Dev team) | 15 September |
A sample of a working plan. Please assume today’s date is 12, September 2021.
While doing this, keep the following in mind:
Whenever possible, add a buffer to your deadlines
If you foresee a task going beyond the buffer date, let all your stakeholders know.
Setup a meeting to see what can be leveraged; for instance, you could cut back on a user story, if ‘feature completion’ is causing the delay
Consider if the release can be postponed by a day or two — it is better to deliver a quality product late than a buggy one on time
Following the plan is fundamental to avoiding the release rush. It will stop you from ending up in a release rush or worse, uncover a high priority production issue after release. All cadence and plan follow-ups are to make sure you’re in a position to anticipate problems.
From experience, I’ve learned that the best way to avoid last-minute anxiousness — or the release rush — is to have complete visibility and ongoing conversations. This practitioner’s guide is designed to help you track the release and encourage open discussions. Feel free to customize it to the needs of your projects and use it in ways that’s most helpful to your work.
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.