Eight lessons from managing agility at scale: part one - delivery operations

Adopting agile practices and an open culture can be a daunting proposition at the best of times. Add to it the complexity of new technology, large enterprise bureaucracy and distributed teams and you have a concoction that can quickly feel cumbersome.

This two-part blog series summarizes some key lessons we learned while building an enterprise-scale data platform based on the principles of Data Mesh for a large healthcare client. We will focus on the challenges around team organization, people management and team culture. 

Challenges of scale

Let’s begin by setting the context of the problem and the challenges we faced:

• Building an enterprise data platform based on a framework like Data Mesh, which is still in its nascent stages

   

• Working with a team of 100+ people spread across 6-7 groups, four different time zones (including US West and India) and belonging to 3+ organizations

   

• We were dependent on slow and bureaucratic IT infrastructure teams, although we were part of a recently acquired digital incubator

   

• Coordinating efforts between teams with data ownership upstream and downstream, which had different reporting structures

Lessons Learned

What’s the weakest link?

You’ve probably heard that a process is as good as its weakest link. As you go about solving the first problem of populating your data platform, ensure that you have done a value stream map that starts with the upstream sources and traces steps even beyond your data platform, right up to your end consumers.

Fig 1. How to address the value stream efficiency

The most important part of this exercise is to create collective ownership among the disparate groups and to move away from finger-pointing to collective ideation and issue resolution. This requires a strong mandate and lean-in from executive leadership, who play a key role in creating an environment where collaboration and communication can flow effectively.

Team structure and hand-offs

It’s tempting and probably necessary to create specialized teams when you reach a certain scale. However, if the team boundaries and composition (i.e. skills) are not addressed appropriately, you will end up in a scenario where the inter-team dependencies will cripple progress. The fact that teams and skillsets are distributed across time zones also does not help. We recommend setting up teams with the following principles in mind:

• Team size not exceeding 12-15 members

   

• Interactions limited to 2-3 other teams

   

• Clearly defined collaboration patterns between teams

   

• Prefer multi-skilled roles to limit hand-offs between teams

Fig 2. The team topology effective for high complexity efforts

Managing scope

The age-old problem of scope management rears its ugly head when you have many stakeholders and teams and if you don’t have a central product management team. The resulting chaos can lead to constantly shifting priorities and impedes team motivation.

If you see this happening, then try these measures:

• Put in place a strong product management team that also works closely with the teams on the ground

   

• Provide technical support to product management teams (mix of product and platform thinking)

   

• Say 'no' to non-aligned scope. There is a thin line between being ‘agile’ and ‘fragile.’ Effective product management teams know when to stand their ground and filter the noise for the teams

Fig 3. Effective product managers help filter out the noise by controlling the intake process

Effective onboarding

We managed to address the operational areas (laptops, access, etc.) of the onboarding experience by soliciting support from a dedicated operations manager. However, the path to effectively transferring knowledge about the actual work was a lot more difficult for a complex and growing platform. Here are some solutions that’ll help reduce the complexity:

Start off by establishing the right onboarding practices and accounting for reduced velocity during onboarding cycles.

Succession planning and rotations

Large-sized teams that have been around for a while need a proactive approach to succession planning and people rotations. When done consciously, the benefits of this approach leads to higher motivation, better growth for individuals and an infusion of fresh ideas. As a leader, you will constantly struggle with maintaining the balance of retaining past context with the aspirations of the various individuals on the team.

Addressing the whack-a-mole bottlenecks, prioritizing the highest priority items, structuring for effectiveness and greasing the wheels for a better team experience were some of the initiatives that helped us bring better operational effectiveness in high complexity team settings.

In the second part of this series, we will dig deeper into practices that are instrumental to building high-performing teams.