Sustainability and ESG (Environmental, Social, and Governance) are now essential for financial services firms to operate in most markets, driven by climate change awareness, failures to meet Paris Agreement targets, greenwashing concerns, sustainability regulations (e.g. Sustainable Finance Disclosure Regulation (SFDR) in EU, Sustainability Reporting Standards (SRS) in the UK), and consumer demand for ESG and sustainability-aligned products.
Corporations, institutional investors, insurers, asset managers, banks, ESG ratings and data services providers, and government oversight entities all play crucial roles in the ESG and sustainability value ecosystem within the financial services sector. These interconnected participants collectively work to meet the growing demand for ESG and sustainability solutions, including ESG products like green and social bonds, low-carbon products, and ESG-focused funds. They also engage in ESG mandates, risk assessments, ESG ratings, and climate risk stress testing, reflecting the comprehensive efforts required to address ESG requirements across the industry.
Robust ESG and sustainability data is critical
Access to robust ESG and sustainability data is crucial for financial institutions to achieve their sustainability goals. This data is essential for accurately assessing risk, meeting investor demands, and fulfilling regulatory requirements for all firms in their portfolios and those they finance. Furthermore, this data must be continuously monitored and updated, safeguarded against misuse (e.g., preventing insider trading), and accessible to a diverse range of stakeholders across the financial ecosystem.
To meet ESG and sustainability requirements, financial services need a diverse array of data sources, including both structured and unstructured, non-traditional (often referred to as 'alternative' data) from external providers as well as from internal systems and databases. As new data sources emerge, they must be swiftly evaluated and integrated into the decision-making framework to enhance asset coverage and maintain a competitive edge.
Challenges with external sustainability and ESG data
Strong interest in the sustainability data to meet the ESG criteria has resulted in myriad players offering ESG-related data services — from numerous rating providers (e.g. MSCI, Bloomberg, Morningstar) to specialists in the sustainability data (e.g. Sustainalytics, Clarity AI, Trucost and many others), and various other alternative data vendors, including supply chain analytics providers. New sustainability data providers continue to emerge, as new data sources become available and technology is making it easier to collect, analyze, and interpret sustainability data worldwide.
External providers' data often suffers transparency, accuracy, comparability and methodology issues, relying on backward-looking, subjective interpretations with weak governance. Regulations like the UK's Sustainability Disclosure Requirements (SDR) further drive the need for granular data for individual holdings in index-tracking funds, rather than only at the portfolio level.
This forces firms to obtain sustainability data and ratings from multiple external providers to complement internal sources. Calstrs' delayed 2023 climate report exemplifies these challenges, with inaccuracies stemming from using different providers' data for its $331bn portfolio's carbon footprint.
Robust data processes spanning sourcing, quality assurance, transformation and reporting are crucial to address external data quality concerns.
Hurdles for ESG and sustainability data initiatives in financial services
Financial services firms often encounter significant challenges when embarking on data initiatives aimed at addressing sustainability and ESG objectives; that not only slow progress but also undermine the efficacy of ESG use cases. These challenges highlight broader industry trends, emphasizing the need for enhanced data governance, streamlined processes, improved cross-team collaboration, and the adoption of modern ways of working with a strong emphasis on data literacy.
A common misconception we often see is that these data challenges can be resolved solely through technological solutions. Firms often invest in new technology stacks and cutting-edge data platforms, believing these will address the underlying issues. However, true transformation requires systemic changes involving shifts in culture, organizational structure and redefining roles around data.
In many firms regulatory reporting teams typically handle ESG compliance, while commercially focused areas develop ESG and sustainability solutions tailored to their needs (e.g. portfolio carbon intensity calculations, green bonds, etc). A major challenge arises from the disjointed handling of data activities among different teams within the organization. For instance, we often see analytics teams spend excessive time navigating data access issues instead of focusing on building innovative solutions, resulting in bottlenecks, delays and increased error potential due to poorly defined and coordinated responsibilities. The separation between business and IT teams exacerbates knowledge silos, redundant work, and data inconsistencies stemming from inefficient data cleaning and transformations.
Additionally, the reliance on external data sources for ESG introduces further complexity and risk. Firms often depend on third-party data providers for critical alternative data, which can be unreliable or inaccurate. This dependence leads to data discrepancies and inadequate data quality checks, compromising the integrity of ESG and sustainability solutions. Onboarding new alternative data sources can also be a lengthy process, taking several months and adding to the delays.
A lack of collaboration within the solution development lifecycle poses additional challenges. Limited coordination and communication among software developers, data engineers, and analysts result in misunderstandings, duplicated efforts, and inefficiencies. This fragmented approach slows development processes and increases the likelihood of errors and inaccuracies in final products.
Moreover, traditional waterfall project management methodologies prevalent in financial services are often ill-suited for complex data initiatives. These methodologies prioritize rigid processes and strict adherence to predetermined plans over customer value and adaptability, stifling innovation and limiting the ability to respond effectively to changing requirements and priorities.
Finally, disparities in data literacy across teams exacerbate these challenges. While some individuals possess deeper expertise to support data initiatives, others lack fundamental understanding, leading to communication barriers and misunderstandings.
Unlocking the potential of Data Mesh for sustainability and ESG
Addressing ESG and sustainability initiatives in large financial services firms is inherently complex. Centralized data management and siloed ‘shadow’ data sources across departments and organizational boundaries exacerbate these challenges. Data Mesh offers a solution by embracing decentralization and aligning strategies, processes and organizational structures accordingly.
Introduced by Zhamak Dehghani in 2019, Data Mesh is a transformative framework that addresses the complexities of data management in large enterprises. It is not a one-size-fits-all solution; organizations must adapt it to their unique needs. Data Mesh represents a technological and organizational shift, emphasizing product thinking and necessitating changes in team structures and new roles.
Data Mesh is built on four core principles:
Domain ownership. Shifts data ownership from a centralized model to domains that manage their data as part of their business area.
Data as a product. Treats data as a product, applying product thinking, and ensuring it is high-quality, discoverable and accessible.
Self-serve data platform. Provides infrastructure and tools enabling domain teams to manage and share their data easily.
Federated computational governance. Establishes a governance model that ensures data interoperability and compliance while allowing domain autonomy.
These principles collectively help firms navigate the challenges of ESG and sustainability data initiatives. By improving collaboration, reducing data silos, and increasing data interoperability and trust through consistent policy enforcement, Data Mesh empowers data teams and accelerates the delivery of initiatives through reusability and clear ownership.
Initiating Data Mesh transformation
Launching a Data Mesh transformation demands a strategic approach to maximize its potential. Instead of a big-bang rollout, we recommend a phased implementation prioritizing rapid value delivery and establishing foundational elements for future scalability.
We strongly advise starting the Data Mesh journey with small-scale initiatives to rapidly learn and apply new concepts thus ensuring tangible value is visible and delivered early to get the buy-in across the organization. Adopting a mindset that allows for learning from mistakes and taking incremental steps forward is crucial, which can be achieved through small iterations and avoiding significant disruptions.
Forming cross-functional teams is a must to start the journey with Data Mesh. These empowered teams serve as the linchpin, bridging domain expertise with technical know-how. Cross-functional teams enable sharp focus on specific use cases such as ESG, building a culture of ownership and accountability. By adopting the "you build it, you run it, you own it" mindset, teams accelerate innovation, while mitigating inter-team dependencies and prioritization challenges. Notably, there is no need to restructure the whole organization all at once — we advise starting with one team, letting them adopt new practices and then scaling out.
Assigning end-to-end ownership of data products to cross-functional teams aligns them with specific domains, ensuring a deep understanding of business requirements and priorities. We advocate for the identification of crucial data products within a few strategic domains to kick-start building the data platform's foundations. Adopting the ‘thin slicing’ approach (Figure below), by prioritizing essential functionalities, needed for selected use cases, over exhaustive technical functionalities accelerates the delivery of tangible value to the organization, minimizing the risk of prolonged timelines and stakeholder disillusionment.
Creating data products for domains not yet onboarded is sometimes necessary to address immediate needs, this will require assuming ownership by domains that already exist and; at a later stage transferring those products to correct ownership as domains come onboard. Simultaneously onboarding multiple domains is impractical without established foundations and governance principles.
Cross-functional teams bring together business stakeholders and experts in data infrastructure, software engineering, UX, and data analysis, delivering tangible business value. However, to enable greater focus on specialist knowledge within individual domains, we advocate for the establishment of a platform enablement team. This team should collaborate with initial use case teams to develop reusable templates and a scalable platform.
The platform should serve diverse user groups, including engineers seeking flexibility for technical innovation and analysts requiring more guidance for areas such as data visualization. Incorporating security and governance by design ensures compliance with organizational standards for all platform-developed products. By adopting an incremental approach, the platform team can prioritize functionality based on the needs of selected use cases, effectively managing scope.
The platform should empower teams by achieving a delicate balance: granting them sufficient autonomy for rapid delivery while also maintaining adequate governance and processes to ensure compliance and the creation of high-quality products. This balance enables innovation and quick value delivery while mitigating risks and ensuring consistency across the platform.
Creating an attractive and efficient development environment not only accelerates value delivery but also increases data product team satisfaction. Demonstrating speed and agility in developing solutions will encourage adoption and showcase the benefits of the approach.
The platform team also plays a vital role in facilitating knowledge sharing among cross-functional data product teams. This collaborative effort lays the foundation for the ultimate goal: a self-service data platform.
We must view our data, use case and platform as a product, keeping customers at the core of the development process. This entails adopting product thinking and agile practices (per Figure below), involving all stakeholders to shape and explore problems and solution space and determine the smallest piece that can deliver immediate value. Once development begins, follow an iterative approach and seek customer feedback regularly to ensure alignment and evolve and scale the data products.
Embrace starting small and change without fear. Overreliance on exhaustive plans can prove counterproductive; data product teams may have progressed or improvised solutions by the time such plans are implemented. This can lead to unnecessary disruptions and forced migrations. Instead, embrace an iterative approach that allows for flexibility and adaptability.
As you develop the initial data products, future scalability should be a primary consideration. With multiple cross-functional data product teams emerging, it's recommended to establish foundational templates for development, covering aspects like ingestion pipelines, CI/CD processes, governance and security from the outset. This will streamline processes and avoid unnecessary complexities, helping to create reusable, easy-to-use artifacts that will enhance the productivity of multiple teams going forward.
Data Mesh introduces the crucial concept of federated governance, integrated from the outset in the approach we advocate. Our data product template incorporates elements of data security, governance, and access by automatically registering data products in a data catalog. This ensures alignment with company security and compliance standards, facilitated by subject matter experts within cross-functional and platform teams.
This approach empowers teams to automate governance processes, ensuring compliance with organizational standards from the start. Instead of governance teams manually pulling information and performing routine checks at the project's end, policy adherence is seamlessly integrated and automatically enforced. Consequently, the central governance team can focus on policy creation and automation, shifting from enforcement to a more collaborative and strategic role. This transition promotes a culture of safe data use while promoting innovation.
Unlocking continuous innovation to drive value
Embracing the core principles of the transformative Data Mesh framework allows financial services firms to navigate the data complexities of ESG and sustainability initiatives while unlocking continuous innovation.
As sustainability and ESG objectives take center stage in strategic priorities, integrating AI-augmented approaches and Generative AI offers profound opportunities. These technologies will empower firms to explore vast repositories of supply chain data, uncovering insights that drive substantial progress toward sustainability goals. However, maximizing the potential of Generative AI requires significant shifts in managing data, processes and talent to create value within enterprises.
Data Mesh provides a robust foundation for these innovations, ensuring progress is not impeded by outdated data management approaches. Here, innovation and sustainability converge to create a lasting and transformative impact on both the industry and society at large.
