By Keith Schulze and Kunal Tiwary
Consider your favorite sporting team: each person brings a unique set of skills and experience to the team and plays a specific role. Powerful and accurate, quarterbacks call the shots on the field, while linebackers defend with strength and speed. Versatile midfielders connect the two and keep the ball moving smoothly. To be a successful team, these roles and skills must work together seamlessly.
Data teams work similarly. As an integral part of larger processes, a successful team will bring together complementary skills, experience and knowledge to deal with critical questions such as:
What new markets, services, or cost optimizations is the business embarking on?
How could data help our people make these decisions?
Is this data available to key decision makers in a form that is understandable?
As the modern data value chain evolves, organizations are making a fundamental shift in how they think about data – and how they build teams around it. As you focus on democratizing data across your organization, think about how you can align your internal structures around the people who make up your data teams. Here’s what we have found useful in building data teams for specific products.
Forming effective teams
Splitting domains or areas of interest is a great way to start identifying what teams you’ll need to form. Look for areas of high cohesion – where elements are closely related to each other and have a common purpose – and low coupling – modules that work independently of each other – between domains. For example, Netflix maps some of its domains and data products as follows:
Subscriptions: forecasting churn and helping predict which customers are likely to cancel their subscription
Content: content recommendations and ranking
Player: client related statistics
Payment: fraud detection
Much like software teams, data product teams own their products collectively. Each product should have a nominated product owner who acts as the team’s ambassador and key communicator to stakeholders and other data product teams. They drive the product roadmap and lifecycle, communicating expectations and facilitating collaboration.
Elements of a winning data team
Developing effective data products requires a specialist team with a set of multidisciplinary skills, experience and knowledge, including data engineers, data scientists, data product managers, data UX researchers and analytics engineers.
One of the most effective ways to build your data team is to develop roles that focus on specific aspects of the product's infrastructure, development and lifecycle. Each role brings a different set of skills, strengths, and approaches needed to create value from data. The nature of the data product you want to build will dictate the roles you’ll need.
It’s important to note that a role is distinct from an individual who plays a part in your team. In fact, some people may fall under multiple roles. You should consider the roles needed to build the product, and those that will operate and maintain it. Different teams will:
Create data products and make sure they’re delivered through reliable data pipelines
Use data products and combine them with advanced analytics to create new business value
Ensure data products are dependable and run smoothly
In some organizations it’s also common to see platform teams with specialized knowledge in certain technical competencies such as infrastructure or data science. This helps reduce the cognitive load on a data product team, because team members don’t need to be specialists outside of their skill sets.
Roles in practice
Consider a scenario where you need to build a new financial services offering for a retailer. Your goal is to serve aggregated customer credit history information to an external entity, safely and securely, to help them make credit lending decisions.
The high-level requirements include:
Ingest credit history data from several internal data sources
Transform and aggregate credit history data into a format that supports credit lending decision making
Provide a secure API to serve real-time requests for the aggregated credit history information for a customer based on a unique customer identifier.
The cross-functional requirements include:
Credit history data must not be older than one day
Data should not leave a secure network environment (i.e. be on the public internet)
Sovereignty and governance of customer credit history data must remain with the retailer. Any data transferred to the external lending provider should be audited and governed by policies for its reuse and handling.
Let's consider some of the roles you will need to build and evolve the customer credit history data product into the future.
Product owner: The product owner is accountable for maximizing your data product’s value. They also play an important role in supporting the team at key decision and prioritization points. They will help prioritize which insight should be built first based on the return of investment of the feature and the effort involved.
Business analyst: they play a vital role in understanding and aligning the value of your data product with the needs of customers and the wider business.
Data engineer: they build the pipelines that source data from several internal systems, and transform and aggregate the data into a form that supports credit decision making.
Infrastructure engineer: they build reusable and scalable infrastructure around the data product to facilitate reproducibility, continuous integration/deployment and automate as much as possible.
Backend engineer: they build business logic in the form of data APIs to make data integrations with UI, other products and visualisation tools easy.
Quality assurance: accountable for maintaining data and product quality. This role is essential to build trust with the customer.
Data science: depending on how well-defined the credit decision making process is and whether we know what data is required, this role may be required to help with this understanding.
An important note on security
Since you will be sharing potentially sensitive data across different organizations, security is a critical concern for this data product. While it’s vital that there is someone in the team leading on security, it should nevertheless also be viewed as the whole team’s responsibility – and you should build security into your product with support from a core security function in the business.
Remember, you don’t need dedicated people for every role. You might have experienced team members who fit many roles, or some of your core platforms might play a supporting role. For example, you might ask someone to be a security and privacy champion and make them accountable for making sure that the team follows good security practices. They can work closely with a core security platform team to run important activities like setting security objectives or running threat modeling sessions.
The skills and knowledge required at each stage of the data product’s lifecycle is different - so the roles you'll need will change along the way.
On the importance of soft skills
While there is overlap in skills in some data team roles, the importance of the below soft skills to round out the roles cannot be overstated.
Leadership: This is vital to establishing a culture of treating data as a product in an organization. Especially if an organization is transitioning from a centralized data team to a decentralized model where autonomous teams are formed around a data product. During this transition, there will be periods of uncertainty and questions about whether the return on investment justifies the effort. Having experienced leaders with a clear vision for how a data product will deliver value for customers and the business can help a team navigate through these uncertain times –and convey the long-term benefits of a data product to the wider business.
- Courage to speak up: Speaking up can take many forms. For example, questioning why we are doing things that aren’t aligned towards our values and goals. Or raising challenges and providing constructive suggestions in a team retrospective (assuming a psychologically safe environment) as the first step towards continuous improvement.
- Communication and storytelling: Effective communication and storytelling help to make data work, which tend to be technical in nature, accessible to non-technical stakeholders, encourage collaboration and improve overall outcomes.
Aligning data teams to organizational structures
Decentralized data teams formed around data products should have their own C-Suite executive and be treated as part of mainstream engineering. Leaders need to set a direction and provide governance – enough to empower the teams and enable autonomy so they can freely experiment and discover. This allows them to deliver true value for customers. The executive’s KPIs should also align with their data team’s goals.
Defining what success looks like for your data team is critical – and goals should focus on enabling decisions (outcomes) rather than the number of dashboards built (activities). For the Subscription domains in the Netflix example above, a success metric might be to reduce churn by 10% over the next quarter – rather than measuring the number of data points acquired to produce that metric.
Holding data team sessions is a great way to communicate these goals and how they fit in with the rest of the organization and build internal understanding of the teams’ functions and roles.
Figure 1: The culture and mindset of decentralized teams
As the managers and coaches of elite sporting teams know, an unbalanced team can fall short when it’s time to perform. We’ve seen many organizations fail to realize the importance of balancing various skill sets when forming their data teams. This often leads to poorly engineered products or user experience – or turning products into a feature factory without considering the use case and customer value.
When you can draw on a broad range of expertise, and have the right structures to get the most value out of each skill set, your teams can target the right problems and thrive. As the famous saying goes, teamwork divides the task and multiplies the success.