How should organizations best go about adopting Data Mesh — and thereby accelerate the delivery of insights at scale?
Data Mesh has been described as a “Socio-Technological paradigm”, because it represents not only a change in architecture and technology, but also a change in how individuals operate and how teams are organized. If people and process changes are ignored, Data Mesh transformation will not be successful.
But as we all know, changing organizations and behaviors is much harder than introducing new technologies. It requires careful planning, consistent communication, and a strong mandate for change. A ground-up approach will quickly demonstrate the benefits of employing Data Mesh principles, but strong leadership from the top is also required to drive adoption across an enterprise and bring everyone along on the journey.
The benefits of both the ground-up and top-down models are captured in a three-phase approach we have described as “Show-Shift-Scale.” This approach enables the benefits of a lightweight introduction, while setting the program up for long-term success as Data Mesh gains momentum in your organization.
Phase I: Show
This first phase is dedicated to showing how Data Mesh enables your technology teams to better deliver data and insight to the organization. To achieve this Proof of Value, a pilot Data Mesh implementation must demonstrate the following:
Data products delivering valuable insights to data consumers.
Just implementing data products does not create a compelling case for adoption. To obtain the buy-in of senior stakeholders, the pilot implementation must show the value of the Data Mesh approach to the users of the data.
Platform capabilities enabling the delivery of data products.
The pilot must deliver some platform capabilities — some self-service data infrastructure and/or federated computational governance — to show the value of the platform itself in building the data product. Otherwise we risk building a data product in isolation, failing to demonstrate the scaling opportunity of technical enablers and a Data Mesh approach.
It is important to recognize that these two goals will compete with each other. It is always easier to build a single data product without a platform approach, but it will be impossible to achieve scale without investing in platform development. Striking the right balance requires strong leadership with a clear vision of the future state.
The Show phase is also about learning how Data Mesh will need to be adapted to work within your company. Each Data Mesh implementation must be tailored to fit your size, distribution, culture, and business requirements. The pilot teams will verify any changes that will be required, and provide a model for how Data Mesh will operate within your organization.
Phase II: Shift
In the Shift phase, organizations shift from a Proof of Value to a Way of Working. To enable broader adoption of Data Mesh, you must first establish the processes, tools, and training required to onboard new data product teams. Providing a “paved path” for the adoption journey will help reduce resistance to change. Overcoming that resistance will require more than effective onboarding processes, however. It will also require coordination, communication, and a leadership mandate to change. Your pilot teams will play a crucial role in gaining the support required to transform the rest of the organization.
Validate these onboarding processes by continuing to activate a small group of new teams. Not only will they provide a good field test of your ability to launch new teams, but you will also demonstrate continuous, accelerating adoption of Data Mesh to your stakeholders and executives. Crucially, this has to happen in parallel with the ongoing delivery of business value.. Pausing the launch of new teams to build these scaling artifacts risks losing momentum on delivering business value. When the business loses interest because they’re not seeing value, funding dries up before the true Data Mesh platform value is realized.
Phase III: Scale
The last step in transformation is Scale. During the Shift phase, you should have completed the planning and preparations required to support widespread adoption. At this stage, many leaders make the mistake of believing the heavy lifting is complete. But scaling adoption will continue to pose a number of challenges that organizations must address to succeed. Continuing to evangelize and demonstrate the benefits of Data Mesh will be required to prevent adoption from wavering. Depending on your organization’s size, distribution, and appetite for change, this transformation can be a multi-year journey. The momentum of a change initiative must be strong enough to overcome a company's inertia toward change.
Key Factors for a Successful Transformation
The goal should never be delivering a Data Mesh. The goal should be accelerating value from insights across the organization, for which Data Mesh can be a compass. Ensure you see value and understand next steps to avoid the trap of “checking the box” on buzzword compliance.
Start with the end in mind. Before beginning a Data Mesh transformation, you should have a very clear and shared understanding of the problem that needs to be solved, and that Data Mesh is the way to solve it. During the adoption, it will be critical to revisit and reiterate the drivers and expectations that launched this transformation.
Remember it’s Proof of Value, not of Concept. Proving that you can do Data Mesh is not the same as proving you should do Data Mesh. To get executive stakeholders onboard, you must demonstrate how those first few implementations are addressing your organization’s challenges in producing and sharing insight.
Don’t rush through the Show and Shift phases. Transforming an organization can be a highly complex process, and can require years to complete. Moving too quickly can increase resistance to change, or lead to failures that can eliminate support for Data Mesh. Stakeholders, particularly those paying, will often press to accelerate efforts and move to scale. However, if you haven’t clearly achieved the objectives of the two initial phases, rushing to scale Data Mesh will lead to failure.
Communicate constant, progressive progress. Demonstrating frequent, meaningful progress to stakeholders and participants is your best strategy for defending against pressures to move faster, and to maintain momentum.
Data Mesh has been proven to accelerate the delivery of insights for large organizations which struggle to unlock the value of data. Successfully adopting Data Mesh requires thoughtful, intentional change of your people, processes, and technologies.