Evolving our release process - fun time lapse video

To help us better understand how our release process has evolved over time, the Go product manager Marco, created the following video and shared it with us. I thought it was a fun way to see how our process has changed over time:

I really enjoyed this video because it told the story of how our build pipelines evolved over time. Note that the pasty environment is our internal dogfooding environment, the dumpling environment is production where are customers run and baklava is a snapshot of production that we use a sanity check for deployments before we go to production. Here are some of the highlights for me:

1. We started our development by running against branch. You can make the argument that this branch was much longer lived than it should have been. We certainly did ;)
2. We went from rather infrequent deployments to production to much more frequent deployments to production. More frequent deployments reduced the risk of deploys since we kept our deltas really small.
3. The practice of deploying to a production snapshot before deploying to production. This was another practice that raised our confidence level wrt deployments.
4. Moving away from our long lived branch to start working from master. While this meant that core Mingle features could be released much sooner to production it did mean that we had to be a lot more disciplined about how we prevent half-baked features from being released to production. We make use of feature toggles to keep half-baked features hidden.
5. Switching over to use AWS AMIs for deployment. This helped to eliminate a lot of flaky failures during deployment.

Of course I still saw some lowlights in that video. Chief among them was the number of failed SaaS builds. We’ve definitely gotten better but we’re still not where we want to be.

The video was made using Go’s new value stream map feature. I’ve already gone on record as saying that this is my new favorite feature in Go.