My colleagues at Thoughtworks and I have always prioritized developer experience when working with clients to help them achieve their business goals more efficiently. Over the years, we’ve seen many instances of developer portals cobbled together from various types of internal document-sharing tools, information radiators, diagrams and the like to help optimize developer experience. While these systems have helped address some challenges engineers and other technical folks face while executing their role, we haven’t seen anything quite like Backstage from Spotify. It’s unique as it’s one application that provides solutions for mitigating many sources of developer friction.
When addressing developer friction, using the right tool for the job is critical to the success of your initiative. In this article, we’ll describe common sources of developer friction and share our experience with how a developer portal platform like Backstage can address your challenges. We’ll describe how far the core capabilities can take you and explore some areas where we’ve seen value in customizing the Backstage implementation to take you even farther on your engineering effectiveness journey.
Lack of discoverability
In any technology organization with more than two teams, finding information, whether service documentation, runbooks on how to respond to production issues or where to find metrics and logs, can often be challenging. These challenges often come from the freedom that independent, empowered teams enjoy — they can choose to put their docs in GitHub directly, generate them to GitHub Pages or use other tools like Confluence or Sharepoint. While teams are free to find a solution which fits them best, the variation in solutions across the organization generates its own friction. As a developer, how do I know where to look for the information I need?
Sometimes entire code bases fall off the map as team membership changes and information gets lost between team transitions. I once worked with a startup where some of their critical systems existed in production with no clear owners. Version control history listed names that were no longer part of the organization, and there was no clear way to trace who possessed critical knowledge about the system. Unfortunately, a production incident revealed this situation. The engineers had to go through an urgent journey of discovery to understand how the orphaned systems worked while things remained broken. It was a struggle to gain enough confidence in their knowledge of those orphaned systems to address the production incident.
Software discoverability, the ability to find information relevant to existing software in an ecosystem, is essential to the day-to-day work of any engineering organization. The lack of it is one of the most significant sources of developer friction and waste. Fortunately, software discoverability is one of the core capabilities Backstage supports. Its Software Catalog feature solves the discoverability challenge by giving engineering organizations a uniform view of all their software, services and libraries, along with metadata like who owns it. It does this through various integrations with SaaS tools like GitHub, Azure, AWS and BitBucket for example. Friction resulting from ambiguity about services and ownership in a software engineering ecosystem quickly disappears. Engineering leaders are empowered to enforce lightweight governance on how services are represented, and engineering teams can provide information about their work in a way their consumers, other internal developers, expect.
Slow to launch new service
Another source of friction is associated with the repeated process of setting up new codebases and deploying them to production as needed. While the choices of tools, libraries and services available can be exciting and fun, it can be overwhelming to vet and experiment with all the new ways to set up codebases each time. For most organizations, it makes more sense to optimize engineering capacity to build software that differentiates them from their competitors rather than waste capacity on the commodity plumbing needed for what should be a repeatable setup process.
In most organizations, there are preferred tech stacks, patterns and tools that help reduce friction by introducing a measure of uniformity. Backstage shines here by offering a Software Template feature allowing one to create templates that generate an initial project codebase. Implementing common requirements in these starter kit templates, such as logging, metrics, telemetry, testing and test coverage, pipelines and even Backstage configurations, provides immediate value to the team. Backstage allows for advanced customization of their Software Template feature such as add-on actions the template creator can run as it executes. This provides value beyond simply creating a new code repository from configuration variables. Automatic registration in the Software Catalog, provisioning of CI/CD resources, or even registration into common code quality, observability, metrics and monitoring systems are all within reach when the Software Template plugin is extended with custom code.
In addition, the Tech Radar plugin, modeled after Thoughtworks’ own Tech Radar, supports engineering organizations in implementing a form of governance via Backstage. The Tech Radar communicates the organization’s tech strategy in terms of languages, frameworks, patterns and tools. Using it gives engineers who want to implement code differently from the starter kit technology stack some guidance.
Over time, a shared foundation for building software reduces fragmentation in the technical landscape— this, in turn, lowers the cognitive load during software operations and maintenance when engineers communicate, collaborate and troubleshoot across multiple software products. In addition, Backstage makes conforming to organizational standards quick and reliable, freeing developers to focus on differentiated value.
Disparate DevOps workflow
Finally, after figuring out who owns what systems, optimizing on initializing codebases and getting them to production comes operations, another area prone to engineering friction. Many organizations follow the “you build it, you run it” DevOps mantra — teams don’t just provide support for the software they’ve built. They’re fully responsible for running it in production. Production systems can get complex and potentially spread out over many different tools and services like CI/CD providers, application hosting, monitoring, alerting, logging and runbooks. These are often hosted or supported by a multitude of vendors, proprietary or open-source, or even built and owned by internal engineering teams. This means that when troubleshooting issues, engineers need to navigate between many repositories, tools, documentation and observability systems to get insights on what is going on with their systems. Many Backstage components come into play here to help engineers eliminate friction in navigating production ecosystems.
Backstage’s Software Catalog integrates with multiple CI/CD SaaS tools to visualize pipelines as code changes move through to production. Once deployed, a Kubernetes integration is available to view deployed services and their health if those services run on Kubernetes. Many plugins exist for monitoring that help engineers visualize and alert on the health of their applications like Datadog, Dynatrace, PagerDuty and Prometheus Backstage plugins. The Backstage TechDocs plugin supports publishing internal technical information. With its docs-like-code style, it supports teams creating runbook-type content and other styles of technical documentation, so software consumers can find data or links to information to debug production systems.
Having all these tools in a central place, Backstage, makes a day in the life of anyone on call or managing production operations easier.
The big picture
Ultimately, optimizing developer experience removes waste, confusion and delay. It allows more room for developer creativity to focus on complex engineering problems that deliver differentiated value instead of trying to survive and react to papercuts or more pressing “keep-alive” issues that disrupt the work day. Besides efficiency gains, these optimization activities support developer happiness and talent retention, a critical component of the current software age in which we find ourselves.
The core value proposition of Backstage is that it’s a developer-centric tool through which an engineering team can provide an efficient, customized, self-service, product-oriented experience to peers and consumers throughout their organization, as-is. It’s extendable through its plugin architecture and customization options to provide a more org-aligned developer portal experience. Teams no longer need to solve problems from scratch to offer a self-service experience — they can leverage the Backstage foundations to integrate their SDLC tooling and processes into one portal. Engineering organizations will no longer waste time on duplicated effort across teams, and engineers will have easy access to the information they need to deliver value for the business.
The road ahead
This article is the first in a series that will explore the opportunities that Backstage provides to streamline how engineering teams deliver software more quickly, with less wasted effort and with higher quality. We'll talk about typical integration challenges and dive into the technical details of how the architecture supports customization to fit your organization’s unique challenges. We’ll also illustrate the types of outcomes that we have achieved through our work implementing Backstage across a variety of organizations.
At Thoughtworks, we’ve worked with many organizations on their engineering effectiveness journey, including Workrise, Spotify, Etsy and Telus. We've developed a modular framework that focuses on six areas within the product and engineering value stream and three organizational enablers to accelerate engineering effectiveness. The developer portal is a foundational component of this framework, through which we can scale knowledge and improve the developer experience. We've partnered with Spotify's Backstage on full-scale developer portal activations, and we're on the ground floor with clients like Telus Digital to support and streamline their developer experience using Backstage.