How can we ensure that engineering work is delivering business impact? That question is one we've been trying to answer at Thoughtworks for some time. There are clearly no easy answers, but a new book — Engineering Excellence to Business Outcomes — offers some unique perspectives and techniques to help build better alignment between engineering and business objectives.
To get a glimpse at the ideas and experiences that shaped the book, we spoke to Sachin Dharmapurikar, one of the co-authors and a Thoughtworker. He gave us an insight on the fundamental principles covered in the book covers and explains his approach to metrics for engineering effectiveness.
Richard Gall: Why do we need engineering effectiveness to business objectives (EEBO) metrics? How are they different from, say, DORA metrics?
Sachin Dharmapurikar: EEBO metrics are crucial because they bridge the gap between engineering activities and their tangible business impact. They establish a direct and measurable connection between the work of engineers and the financial outcome.
By using EEBO metrics, organizations can: justify engineering investments, by demonstrating the ROI of engineering efforts; align engineering with business goals — ensuring engineering teams are working on projects that directly contribute to business objectives; and make data-driven decisions by using metrics to inform strategic choices and resource allocation.
They can also increase efficiency and effectiveness. They do this by improving how efficiently and effectively engineering work is done, organizations can gain a competitive edge and achieve better overall results.
While both EEBO and DORA metrics are valuable tools for measuring software development performance, they serve different purposes. DORA metrics are internal to an engineering organization and focus primarily on software delivery performance. EEBO metrics, meanwhile, are intended to be external to engineering and focus instead on business impact.
It's also worth noting that the DORA metrics are quite specific: they include deployment frequency, lead time for changes, mean time to recover and change failure rate. EEBO metrics, on the other hand are derived from multiple data points,which are often multivariate. They measure the impact of other metrics on the overall program.
Another way to think about it is that DORA metrics help optimize the software development process, while EEBO metrics help show the value of that process to the business.
By combining both sets of metrics, organizations can gain a comprehensive understanding of their software development performance and its impact on the business.
RG: Why is it so hard to bridge the gap between engineering work and business impact?
SD: Bridging the gap between engineering work and business impact is challenging because of a variety of factors.
Traditional metrics often prioritize activity over value. Metrics like velocity and throughout, while useful for tracking engineering output, can mislead when used as the sole measure of success. These activity-focused metrics cannot capture the impact of engineering efforts on tangible business outcomes like customer satisfaction, revenue growth, or market share.
There's a natural tendency to focus on easily measurable metrics rather than meaningful ones. When pressured to show progress, engineering teams may rely on readily available data points that don't reflect the true value of their work, creating "dysfunctional metrics" that create a false sense of accomplishment and obscure areas which need improvement.
Organizational silos and a lack of shared understanding between engineering and business stakeholders further exacerbate the challenge. When teams operate in isolation, using different metrics and definitions, it becomes difficult to establish a clear line of sight between engineering activities and their impact on business goals. This lack of alignment can cause misaligned priorities, wasted effort, and missed opportunities.
Even well-intentioned frameworks like OKRs, while valuable for setting objectives, often lack a robust mechanism for directly correlating engineering excellence practices to business outcomes. This can lead to engineering tasks, especially those related to non-functional requirements, being deprioritized or inadequately resourced.
RG: We often think of metrics as a kind of science — but to what extent would you say selecting and implementing the right ones is an art as well?
SD: When selecting and implementing the right metrics, there is also a significant art involved, despite them being often perceived as purely scientific.
Recognizing the subjective nature of "value" is key to understanding the artistry involved in metric selection. To understand the artistry involved in metric selection, one must recognize the subjective nature of "value" and interpret and judge the translation of activity metrics like lines of code or deployment frequency. For instance, a high deployment frequency, while positive, might not generate business value if the deployed features do not align with customer needs or market demands.
The shift from readily available "activity metrics" to carefully chosen "fitness metrics." This shift requires a nuanced understanding of the organization's goals, context, and desired outcomes, moving beyond simply measuring what's easy to measure. Determining the most relevant fitness metrics, such as those related to customer effort score or time to market, needs a deep understanding of the interplay between engineering work and business impact.
Successfully implementing EEBO metrics relies heavily on effective storytelling and narrative-building. To effectively communicate the value of engineering excellence to stakeholders, it is necessary to craft a compelling narrative, even with robust data and well-defined metrics. This involves translating technical jargon into relatable language, highlighting the connection between engineering efforts and business outcomes, and presenting the information in a clear and engaging manner.
Therefore, while the science of metrics lies in their objective measurement and analysis, the art emerges in the subjective processes of:
Discerning valuable metrics from those that are merely measurable.
Framing technical achievements within the context of business impact.
Communicating insights in a way that resonates with stakeholders and drives action.
Mastering both the scientific and artistic aspects of metrics is crucial for organizations aiming to bridge the gap between engineering excellence and business success.
RG: Metrics measure — but how can we measure metrics (in terms of their usefulness or effectiveness)?
SD: To accurately assess the value or effectiveness of metrics, one must do more than just gather data. They should also consider how well the metrics align with the organization's goals, their ability to provide practical insights, and their influence on team behavior and decision-making.
We advise measuring metrics through the following lenses:
Alignment with business outcomes: Metrics should not exist in isolation but should clearly show how engineering excellence translates into tangible business value. For example, instead of simply tracking "code commit frequency," focus on "time to market" as a metric that directly reflects the business impact of engineering efficiency. This involves:
Identifying key business outcomes.
Selecting metrics that have a demonstrable link to those outcomes.
Continuously evaluating and adjusting metrics to ensure ongoing relevance.
Actionability and Insights: Effective metrics don't just measure; they inform action. Evaluate metrics based on their ability to:
Provide clear and understandable insights into team performance.
Identify areas for improvement and guide decision-making.
Promote proactive problem-solving by setting benchmarks for failures and devising plans to handle them.
Impact on team behavior: Metrics can influence how teams prioritize work and approach problem-solving. It is essential to assess whether your chosen metrics encourage:
Focus on delivering value over simply meeting arbitrary targets.
Collaboration and knowledge sharing to drive continuous improvement.
A sense of ownership and accountability for outcomes.
By critiquing metrics through these lenses, organizations can ensure that their measurement systems are not merely tracking activity but are actively driving engineering excellence and business success.
RG: Who is your book for? Who did you have in mind when you were writing it?
SD: The book primarily targets individuals and organizations invested in aligning software engineering efforts with tangible business value. It's particularly relevant to:
Technology leaders: The book equips tech leaders with the EEBO Metrics framework, enabling them to measure and showcase how engineering teams contribute to the organization's overall benefit.
Software development teams: By illustrating the correlation between engineering excellence and business outcomes, the book provides teams with a sense of purpose, enhancing motivation and alignment.
Businesses undergoing agile transformation: The book addresses the pitfalls of "dysfunctional metrics" often used during agile transformations. It offers EEBO Metrics as a solution to measure progress effectively and ensure alignment with business goals.
The book aims to bridge the gap between engineering excellence and business success by providing a practical guide to designing, implementing, and using EEBO Metrics. It caters to a wide audience, including those at the helm of startups and those navigating large-scale projects.
RG: One of the key issues in the book is bridging the gap between technology and business. While it’s great that to have a framework or set of metrics to help with this, who should ultimately be responsible inside an organization?
SD: I don’t believe one person should bridge the gap between technology and business. We strongly suggest a collaborative approach involving various stakeholders.
Our recommendations are:
Shared vision and communication: Engineering team needs a shared vision and clear communication with business stakeholders as a prerequisite for successful implementation of EEBO Metrics. This suggests that responsibility doesn't lie solely with one individual or team, but requires a joint effort.
Collaboration in metric selection: We strongly recommend selecting success markers and baselines in collaboration with different people involved in and around your team, e.g. agile coaches, tech leads, and project managers, considering industry standards and team dynamics. This collaborative approach ensures that the chosen metrics are relevant and meaningful to all parties involved.
Joint ownership and accountability: Have a RACI matrix to establish clear ownership and accountability for each metric, preventing ambiguity and fostering a culture of responsibility.
We advocate for a shared responsibility model where technology and business stakeholders collaborate closely. This collaboration includes defining a shared vision, selecting appropriate metrics, establishing clear ownership, and ensuring effective communication throughout the process.
Thanks to Sachin for taking the time to talk. You can learn more about Engineering Effectiveness to Business Outcomes on its book page or listen to Sachin and his co-author Dinker Charak talk about EEBO metrics on the Technology Podcast
Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.
