How not to be intimidated by adopting data mesh
Adoption of data mesh continues to grow. From global pharmaceutical manufacturers to leading financial institutions, data mesh is enabling companies to accelerate the delivery of business-boosting insight. But while early adopters are reaping the benefits, others are stuck on the start line, held back by the daunting level of effort and investment they think it will take to adopt an enterprise-wide data mesh. But it doesn’t have to be that way.
Change is hard, and their reluctance is understandable. But in the age of AI, becoming an insights-driven business is more critical than ever. We’ve already seen how generative AI is disrupting business models; ensuring your organization is the disruptor not the disrupted demands a solid data management foundation. Data mesh can help you put that in place, and enable your organization to quickly explore and exploit the power of generative AI.
Why data mesh?
Data mesh is a set of principles that enable independent teams to quickly create data products — high-quality data assets that behave in a well-defined and consistent fashion.
These principles also enable organizations to use a wide variety of tools, and to adapt practices that are attuned to their specific operational needs. These principles are:
Firstly, treat data as a product. Think of it as having an inherent worth and usage within your organization. It should be decentralized and made available to independent teams.
Secondly, create those teams based around where the data is created and resides, rather than around their technical function. For instance, you may want to create a team based on the sales department, as they understand the associated roles and how the data can be useful.
Thirdly, federate your responsibilities for data governance, so your teams are all using it and are providing feedback.
And finally, introduce a self-service infrastructure that creates a common workbench for all the product teams so they are using data in the same way, which means interoperability comes built in.
Every journey starts with a single step…
Adopting these four principles will necessarily require changes in how people work, the processes they follow, and the technology they use. Change on that scale is never easy, especially in a large organization, but having partnered with clients on those change programs, we, at Thoughtworks, have identified techniques that can make their journey less daunting. These include:
Breaking down the process into smaller, discrete steps that show incremental progress to your stakeholders.
Employ a “thin vertical slice” approach that delivers value to data consumers, rather than building technical capabilities from ground up.
Don’t try to change your entire organization at once (boiling the ocean).
Don’t try to get every detail hammered out before you start changing. Change iteratively, evolving your capabilities, your architecture, and your organization over time.
Data mesh delivers more bang for your bucks
When it comes to clinical trials, data is key. A global pharmaceutical company was struggling to provide a comprehensive view of all of their clinical trials. Their goal was to optimize their investments in the most promising treatments, by rapidly identifying those drugs that were unlikely to make the grade. By quickly terminating those trials, the company could maximize their investments in exploring other options.
Their challenge was that this information was difficult to extract from the multiple Clinical Trial Management Systems (CTMS) in place across the business. Prior efforts to solve this problem by implementing an enterprise data lake provided a way to collate large volumes of data, but the desired goal of using that data to provide enterprise insights proved elusive.
At this point, the pharmaceutical company decides to explore data mesh. They began incrementally, building data products to support a clinical dashboard solution, and then began solving problems like trial diversity reporting. That step-by-step approach has ensured they have such robust data foundations that they’ve been able to use data mesh to power generative AI solutions to enable drug discovery.
Three tips for data mesh success
Identify the pain points where your business is suffering due to lack of insight: the places where lack the data needed to make informed decisions or where processes are eating up a lot of time and resources. It could be a regular report that takes hours each quarter to produce, for example. Start by focusing on this one problem and build a data mesh solution that will solve this particular challenge. Other use cases and successes can come later.
Find the champions within your organization who will want to get behind the solution that you’re putting together. Don’t think of it as a technical problem — you’re solving a business problem and providing insight. So rather than a technical team, you need champions from key business divisions and the C-suite. They can drive the project and will see the value when the problem is solved.
Don’t put it off, thinking that the prospect of ‘data mesh’ is too big or expensive. The first project that you carry out will lay strong foundations for the future. You will be able to prove the value of addressing data as a product, and will be able to build on the foundations to address other business issues as you go.
Ultimately, the aspiration is to create an evolving architecture rather than build something that is set in stone forever. Product teams are continually evolving, adjusting and tweaking processes as the business is running. The same is true for data mesh. The platform and processes that you put in place will evolve as time goes on and business changes, but with strong foundations in place, your business and your teams will be able to call up data and act on insight with alacrity.
