Maximize your tech investments with Engineering Effectiveness

Is your engineering organization maximizing its full capability? During more than 30 years of experience driving agile, DevOps, and platform transformations with our clients, we have observed that most are utilizing less than 30%.  

And while most companies have embarked on some form of DevOps, platform and agile transformation, research shows that, while the "elite" organizations continue to move faster, most enterprises’ transformation results are stagnating.

A host of "silver bullet" strategies and schools of thought have emerged that respond to these issues. ‘Developer Experience (DX)’. ‘Engineering excellence.’  ‘Developer productivity.’  Each has its own merits, but ultimately, each addresses only a subset of the aspects of the overarching goal — end-to-end engineering effectiveness. 

What’s been missing is a field-tested and data-driven framework and methodology that combines the various platform technologies, engineering practices, and developer experience approaches under an overarching measurement and governance model. With a holistic framework and methodology, engineering leaders can transform their software value stream to maximize and accelerate the impact of their investments faster.

With decades of experience delivering engineering transformations for organizations of all sizes, Thoughtworks has unique and practical insight into the most important, but often overlooked, areas of focus and the 'hidden' forms of waste and friction that create drag on engineering productivity. Examples of those hidden, but highly impactful, sources of waste and friction that we commonly witness and remediate are:

• Challenges with enterprise-wide testing and quality assurance, such as catching defects very late in the process (or in production), and delays from unreliable, out of sync, and oversubscribed shared environments. 

• Operating model problems, such as poor flow of well-defined work between product and engineering functions, inter-team dependencies driving wait time, and low platform capability adoption due to a lack of technical product management focus and skills, and high degrees of cognitive load for key talent.

• Ineffective knowledge management, including poor or missing documentation, cognitive overload, excessive communication between individuals and teams due to “tribal” knowledge, and a lack of clarity over who owns systems and APIs, etc.

• Highly coupled legacy architectures slowing or stalling efforts to become more “domain oriented”, due to limited experience building modern microservice architectures at scale, and failed efforts to convert to a more modern API approach (e.g. “REST”).

• The "new frozen middle": Teams lack capable engineering leaders and directors who know how to lead and inspire talented engineers and focus on the practices that create real results, not vanity metrics like “velocity” and “code coverage” (yes - those metrics are poor indicators of effective engineering.)  

   

The Engineering Effectiveness framework that follows is the result of decades of experience driving impactful change across large organizations. Unlike the "cargo cult" approaches to DevOps and Agile transformation, this framework was engineered from the ground up to identify and eliminate the most impactful waste and friction in your organization. 

Our goal is to provide a structured, quantitative, and ongoing approach to reducing time to value, increasing productivity, improving developer satisfaction, and ultimately building a more predictable, efficient, and effective engineering organization.

Productivity is hard to measure*. Fortunately, waste isn’t.

To determine the best way to enable and support engineering effectiveness, we first need to be clear about what your organization wants to achieve, and how high-performing, effective engineering serves your key business drivers. Overwhelmingly, the four biggest drivers for adopting Engineering Effectiveness strategies are:

• Reducing the total cost of engineering, typically by increasing engineering productivity, and by identifying skill and capability mismatches to desired outcomes. 

• Acquiring and retaining top development talent, by delivering satisfying experiences and removing the barriers to individual, team, and cross-organization throughput and flow.

• Reducing time to value and rolling out new software quickly to stay ahead of disruptive players and digital native competitors.

• Increasing the predictability of delivery to enable more accurate, strategic, long-term decision-making and planning.

   

The clear thread across these goals is productivity. Organizations want to build and maintain highly-productive engineering teams that deliver the software they need quickly, cost-effectively, and to a consistently high standard. The problem is, from an improvement point of view, productivity is very hard to measure.

We can, of course, measure it by team output and commits. But in the software space, raw output is no real indication of user outcome or business value creation, and traditional metrics like velocity are relative measures, and easy to game. The various definitions of productivity and ways it can be measured make it very hard to increase consistently, and can easily demoralize teams, or accidentally encourage the wrong behaviors.

So, we reframed the issue and shifted the focus away from deceptive “absolute” measures of productivity, and onto all the forms of waste and friction that reduce it. Unlike productivity, both waste and friction are very easy to track and measure (if you know where to look). 

First, all of the high-impact sources of waste need to be accounted for, and then, most importantly, their mitigation strategies need to be prioritized based on their quantified impact on value delivery, adjusted for cost and complexity. Too often, organizations go after the “low hanging fruit” just because it’s easy, when often the impact of remediating it is also low.  This is often driven, in part, by the trap of focussing on what is easy to measure, or already being measured, rather than doing the work to measure what is important, and of high value.

The key to tackling both waste and process friction is visualization. And therein lies the crucial piece of thinking that is the foundation of Thoughtworks’ Engineering Effectiveness solution. If we can make the waste and friction across software and engineering value streams visible and quantifiable, we can empower teams and leaders to reduce or even eliminate them — and in doing so, improve developer satisfaction and become far more effective in delivering critical business value.

* According to Google engineering and many others, productivity between teams, and even within a team, is very subjective and hard to measure consistently over time.  What is incontrovertible though, is that any waste you remove becomes capacity you can redeploy for more throughput, or cost you can recover.  An example of this reasoning from a Google engineering productivity leader can be found here.

Inside the framework

We built our Engineering Effectiveness framework on the specific techniques, practices, cultural norms and operating models that create impact at scale for elite engineering organizations and digital natives, so that any organization can easily and rapidly follow their recipes for high-performance. Here’s how that thinking forms the framework.

The basis for executing with the framework is driven from an end-to-end, quantitative product and engineering value stream and impact analysis. This is the first step to uncovering the real, and often hidden, sources of waste and friction.  By surfacing them across an organization’s engineering operations, we can then identify the most impactful interventions and investments to drive meaningful, lasting, and systemic change.

The framework breaks those interventions down into six focus areas:

• Platform and engineering capabilities: Ensuring that platforms, pipelines, observability and other capabilities that support developers are easy to work with and drive engineering productivity, rather than limiting it or causing daily frustration for teams.

• DevEx and productivity accelerators: Focus on “starter kits”, reference implementations, custom shared tooling and capabilities, built by engineers for engineers. These are designed to improve developer experiences and make it as easy as possible for them to do their job and automate away manual and error-prone tasks quickly and effectively.

• Enterprise testing strategy and enablement: Ensuring that strategies and testing practices are aligned with the challenges of testing legacy, and cloud-native and microservice ecosystems, and ensuring interoperability between them, at scale and across your organization.

• Domain-driven and cloud architecture enablement: Helping organizations succeed with strategic domain-driven design, and craft a platform architecture, and the aligned team boundaries that enable true autonomy, which are needed to improve both the software systems and integration outcomes that serve complex business environments.

• Scaling knowledge and leadership: Improving the quality of documentation and data cataloging, and creating new enabling structures such as Centers of Excellence to improve the accessibility of the information needed to excel as an engineer.

• Operating model alignment and governance: Improving the flow of work, predictability, and time to value through an engineering organizational and leadership structures aligned to outcomes, value-driven delivery governance and portfolio management practices and instilling product thinking at all levels of an organization.

Deploying the processes, governance, and leadership changes required to deliver Engineering Effectiveness at scale in most organizations requires significant strategy and change management capability. For this reason, our Engineering Effectiveness framework incorporates the critical organizational enablers of organizational change management —  communication, structured upskilling of both individual contributors and leaders, and talent management. 

While many organizations have some of these capabilities, we’ve recognized that effective change in engineering organizations requires engineering leaders to co-create together with their Organizational Change Management and HR functions, both for credibility with engineers, and to successfully navigate the many unique considerations for engineering culture change and communication. 

In some organizations, communication methods, culture, and a lack of disciplined, data-driven strategy practice may actually be sources of waste and friction, a common example being unclear chains of command that limit effective communication and agility. To elevate performance, formalized upskilling and enablement programs — rather than best effort or self-service learning resources — are often needed to build talent quickly and uniformly across the different collaborating areas of engineering organization. 

Talent management and recruitment support can ensure that organizations build a compelling employer brand to attract the right kinds of technical talent that can be productive in the organization's unique technology environment.  It’s especially important for engineering leader hires to have the requisite skills and experience to drive organizational transformation. 

We have seen that many great engineers and engineering leaders who have worked in digital native organizations “know what good looks like” but often do not know how to lead the change needed to create those highly effective environments they have previously worked in.

Getting to the core: Effective, aligned, autonomous engineers

Crucially, the framework isn’t a “one-size-fits-all” plan for improving engineering effectiveness in any organization. Each environment will present unique challenges, and require different interventions to be prioritized for maximum impact. Current maturity level, business priorities, and capacity for change must all be considered. In some situations, we find it most valuable to initially address only one or two of the framework’s six major focus areas. In other situations, we might address all six areas simultaneously, starting with the highest impact initiatives, refining and deepening focus over time.

But whatever interventions are required, every time we apply the framework, we’re always working toward the same core goal — creating effective, aligned, and autonomous engineering teams across the organization.

By applying our expertise to help organizations visualize waste and friction across their processes, pinpoint exactly where and how to intervene to solve those issues, and create key enabling capabilities quickly, Thoughtworks helps create engineering environments where:

• Decision-making is federated across the organization to keep teams agile and effective, while staying aligned with business strategy.

• Costs are minimized because there are fewer errors and time-wasting barriers to delivery.

• Delivery is truly “agile”, accelerated, and more predictable, thanks to the reduction of friction across the software value stream.

• Engineers enjoy their work every day, clearly understand their goals, and are empowered to achieve them in logical, friction-free ways.

Tips for effective transformation

Our framework is a powerful tool for enabling engineering effectiveness, but it takes more than just the right plan and structure to ensure long-term success. If you’re planning, or already executing your own engineering transformation, here are three tips to keep in mind:

#1) Make sure you know ‘what good really looks like”

As you take steps to make your engineers more effective, it’s important to be clear about the state you’re trying to move toward. Engineering best practice is constantly evolving, and if you’re trying to move toward an outdated definition of “DevOps' success", you won’t create nearly as much impact as you could be getting, for the same effort, knowing what the state of the art really is.  

Honestly, high-performing engineering organizations haven’t been even using the term DevOps for years now — if you are still talking about a “DevOps” transformation you may be indexing on a 5+ year old model of success.  You don’t need to tread the same steps the leaders did, you can leapfrog many years of effort by knowing what “high performing” looks like today.

#2) Don’t underestimate the impact of leadership and governance

When you’re trying to improve the effectiveness of engineers, it’s tempting to focus your attention primarily at the individual contributor and team level. But many factors that limit engineering effectiveness are created far higher up in the organization.

Engineering effectiveness demands significant cultural, process, and organizational transformation — all of which need to be led from the top. Without the vision and support of leaders at all levels of the engineering organization (and sometimes beyond), it’s virtually impossible to drive significant improvements in engineering effectiveness. 

“Bottom-up” or “organic” transformation is also needed, but grassroots efforts usually stall without alignment throughout the entire chain of leadership. The most often overlooked factor is the engineering version of the famous “frozen middle” — engineering directors whose incentives and skillsets do not match the change they are being called upon to lead. 

The “connective tissue” of director-level and “staff engineer” leaders must have both deep technical and architectural knowledge, as well as people leadership and organizational improvement experience. This ensures that the senior leadership vision informs the strategies that need to be executed at the team level and that feedback from where the work gets done gets incorporated back up into top-level strategy.   

This may be the single biggest factor we have witnessed in lagging or failed transformation efforts — a frozen (or even missing) middle level of leaders.  Even when a courageous senior engineering leader is driving change and the engineers on the ground are innovating and agitating, experienced and effective mid-level engineering leadership is the critical factor to keeping all the transformative energy constructive and pointing in the same direction.