Evolutionary Architecture, microservices and Data Mesh have all been widely explained and explored as individual technical approaches, but they have rarely been explored together, as things that can support and complement each other as part of a broader data modernization project.
In this blog post we’ll take a closer look at the similarities, differences and correlations between the three and explore how together they can support an incremental process of change that is both flexible and robust.
Microservices architecture enabling organizations modernization
For years, countless organizations have been successfully transforming monolithic systems into microservices-based architectures as part of their modernization journey. They have typically been doing this using a distributed systems model where a group of small independent services communicate over well-defined APIs and work together to achieve successful business outcomes.
The way we modernize enterprise software applications will fundamentally change as we move towards “Evolutionary Architecture.” This is where an iterative approach is used — based on a guided, monitored and incremental change across multiple dimensions — instead of simply re-architecting every part of a large monolithic system.
Gradually modernize your centralized data platform with Data Mesh
Data Mesh is a decentralized approach to data architecture, originally defined by ex-Thoughtworker Zhamak Dehghani (see the original article). It’s based on an analytical data architecture and operating model where data is treated as a product and owned by the team that is closest to the data.
An evolutionary approach has worked well for the gradual modernization of applications with microservices and is also suitable in a data-driven context.
Similarities and differences with Microservices
Data Mesh is based on decentralized governance and on a distributed systems model; this approach is similar to the one advocated by Microservices; by embracing distributed architecture and decentralised governance data mesh is doing to data warehousing what microservices did for monolithic applications.
Data Mesh like microservices is able to identify distinct, independent domains according to how the data needs to be transformed, governed and served, in the same way data products can be implemented by separate parts of the business aligning with the definition of the strong bounded context principle.
Many analytical data products originate or evolve from operational systems microservices. Source-aligned data products, for example, get their data from the operation system with which they are collaborating. A “User Profiles” data product might simply originate from a user onboarding using an existing microservice application.
One of the fundamental differences between Microservices and data products is the relation between the code and its data.
As stated by Zhamak:
“Data and code coexisting as one unit is not a new concept for people who have managed microservices architecture. The evolution of operation systems has moved to a model in which each service manages its code and data, schema definition, and upgrades. The difference between an operational system is the relation between the code and its data.”
“In the case of microservices architecture, data serves the code. It maintains state so that the code can complete its job, serving business capabilities.”
“In the case of a data product and data mesh this relationship is inverse: code serves data. The code transforms the data, maintaining its integrity, governing its policies, and ultimately serves it.”
Is Data Mesh right for your organization?
The goal of this article is to encourage constructive conversations around Data Mesh adoption using an evolutionary architectural approach. However, before you embark on that journey towards a decentralized model you still need to take some important considerations into account.
Data Mesh is not just a new approach to data architecture; it’s a completely new way of thinking about governing, managing and operationalizing data. It’s not a silver bullet to solve all data-related challenges. It’s a change in approach that demands discipline and long-term commitment before it can be applied at scale. It also requires coordination to ensure a balance of federation and governance that can properly align, enable and support new data product teams.
Conclusion
Building on the foundations of microservices architectures, adopting data mesh for your organization to modernize your data platform can propel your organization forward. However, it’s important to recognize the commitment required to get there and the scope of change the move to Data Mesh demands.
If you’d like to learn more about how you can apply the Data Mesh approach for your organization, and discover how Thoughtworks can help you navigate the challenges on the way, please get in touch with us at contact-au@thoughtworks.com.
Aviso legal: Las declaraciones y opiniones expresadas en este artículo son las del autor/a o autores y no reflejan necesariamente las posiciones de Thoughtworks.