Scaling mobile delivery

Episode transcript

​​[MUSIC PLAYING]

Scott Shaw: So welcome, everybody, to the Thoughtworks technology podcast. We're going to be talking about mobile delivery in this segment. I'm Scott Shaw, and I'm one of the hosts of the technology podcast. And I'm joined today by a few other people. Birgitta is my co-host. You want to introduce yourself, Birgitta?

Birgitta Böckeler: Mhm. Yeah. Hi, everybody. I'm Birgitta Boeckeler. I'm a technical principal with Thoughtworks in Berlin, Germany. Thanks. And our guests today are a couple of prominent mobile developers. We'll start with Alexandra. Do you want to introduce yourself?

Alexandra Lovin: Oh, sure. Hi, everyone. I'm Alexandra Lovin — I'm a software engineer who has actually transitioned into the mobile space, so at the early stages of it, let's say, emerging as a technology. And I'm joining from Bucharest, Romania.

Scott: Thanks. And Andres?

Andres Kievsky: Hey. I'm Andres Kievsky. I'm originally from Argentina. I'm a principal mobile developer here in Thoughtworks Australia in sunny Sydney.

Scott: Thanks. We even have sunny Melbourne today for a change. Andres and I are both in Australia, despite our accents. One of the things that I've noticed over the last few years is a growing interest from some businesses in mobile. We know that the percentage of internet traffic that's attributed to mobile devices has been growing steadily over the years. And I think it's at 60% right now. And we've seen a real boost in that. It's accelerated, I think, over the COVID years with remote working in isolation and things. 

And I've noticed a trend that goes along with this. As businesses are trying to accommodate that, more and more of their customers are on mobile. And I think they're most engaged customers typically are on the mobile apps, on the native mobile apps.

Businesses have tried to keep up with this. And I see them hitting up against the limits of their mobile applications that they may not originally have imagined would be so important to their business. And part of this is in how do we get so many teams — how do we treat a mobile application the way we've treated other applications for years as a parallel activity across many teams? A lot of mobile applications weren't set up for that. 

So I see customers asking me and lots of businesses out there asking how do we structure our delivery? How do we structure our architecture? How do we adapt our application to be able to develop more quickly with more teams, and scale up that delivery process? So that's kind of what I was hoping we could talk about today.

But let's, first of all, kind of just get a sense of what is the state of the industry. Maybe, Andres, if you could start us out, how do you see the state of the industry in terms of large-scale mobile delivery?

Andres: I think we can say that mobile remains a little bit of a mystery in some sense. There's companies that are all in, and they're ready to treat their customers that are coming majorly through mobile as the customers that they are, as the mobile users. And they partition, and they create more delivery teams that reflect that. And then there's others that treat mobile as a sort of second-class citizen. That is a reflection of the way that they treat web in their own companies.

And so we see all sorts of approaches. And we see many places where there's been a lot of pain trying to move from that first view of mobile being something that is really web in a little screen to a first-class experience that is of value to the customer, that brings a new and intriguing set of possibilities to our customers — and I think that reflects even in the way that they partition their teams and they create their development teams in-house.

Scott: What are some of the things-- I've heard this said also, that it's important to treat your mobile application as a first-class citizen and not as just an add-on to the web. But what does that mean? What are the things that are unique or that you'd want to do differently with a mobile application?

Andres: There's a very — so your customers have a very close relationship with our devices. So you need to be aware of, say, things like the UI conventions that they expect to be working with, such as battery usage, data usage, and other things in that terrain which are not necessarily taking into account when someone is targeting web. So there's also great challenges around security and privacy that come with that.

So the fact that the customer is so close to your product brings a whole set of challenges that are fairly difficult to tackle and that don't exist in the web. Right? Even the way that we QA mobile is different. And the fact that we have two big players like Google and Apple keeping tight control of the platforms makes it so that we have a whole host of other concerns and external concerns that you don't usually have to think about when you're delivering in web, anyone delivering other back-end systems.

Scott: Yeah. I know that there's some intermittent connectivity sometimes a problem as well. You want your app to start up right away every time, even though your connectivity may not be what it is on a desktop.

Andres: Yeah, absolutely. The possibility of using your app disconnected and the way that you treat the variety of devices that come in particularly in Android, fragmentation is a classic problem when it comes to Android devices, different screens, different generations of mobile devices with different capabilities. And yet your customers will absolutely demand that their apps work correctly in their devices.

And they will feel very close to the day-to-day usage of their devices. The way that they work, the way that they interact with them is so close to their heart that you need extra care. You need extra specialization to deal with that. And you need to make sure that your delivery teams are aware and they have the specialists there to service these challenges from disconnection of operate-- being able to operate your devices in a disconnected fashion to being able to continue using the right UI conventions for the right device.

The other part is your devices are also very well aware of where you are, what you're doing, and that brings up security privacy concerns but also an extra set of streams of data and information that your app can use that your web and other parts of where your code might be running are not. They don't really have that capability. So all in all, this sort of creates an environment where you have huge challenges in terms of the way that you integrate all this technology into one place but also a degree of value creation that is difficult to match, particularly when we're so close to the user and so able to produce all these bits of data that make for really a wonderful experience, right?

So the main idea here is because of the proximity of the user of your customers to their devices, you need to make sure that you're not adding to the friction of their everyday work. And that can happen in many places, right?

And from having a proper UX process to making sure that this very clear and well-defined QA processes to ensuring that you have this proper specialization to deal with all this is something that requires a lot of thought and across a view of the creation and the development of applications that goes beyond just a simple mechanistic view. Right? We want to think about an organic view where teams work together to not just create applications but also think about experiences, and think about the person first, and the devices, and the software, and the cool technology second.

Scott: There's a lot to unpack there. One thing you mentioned was the proliferation of devices and the fragmentation of devices. And Alexandra, you mentioned that you've come from a general software engineering background.

And I wonder what — some of the engineering practices that we apply every day in, say, back-end development may not be as easy to implement with mobile, like continuous delivery. Have you run into-- did you have to adapt your standards when you went to mobile?

Alexandra: Yes, absolutely. It's a slightly different problem space in mobile than in other software engineering areas.

First of all, because of the mentioned fragmentation, which is not only between the two major platforms, Apple and Google, but also a lot inside of the platforms, there are different devices with different hardware specifications, different strain sizes, different operating systems. And at the same time, they do have different, let's say, behaviors in terms of virtual devices, emulator, simulators, and what you can actually do with physical devices.

Because there are scenarios, especially when interacting with, let's say, hardware, hardware accelerators, or external sensors, where the need of actually having a physical device and the ability to also perform integration and testing on a physical device is still required. There are some services such as device labs or farms that can help you with testing on physical devices without having to have your team actually have and own the whole set of devices. Plus it's very important to know the usage, the target of your product, and basically from the multitude of devices that are out there to target those that are most commonly used so that you can tackle most of the paths. This is, let's say, on the testing side. On the actual, let's say, development cycle in terms of continuous integration and continuous delivery, we have seen, let's say, a pattern where the pipeline is also moved into the cloud, whether as maintained by the team or using services that are self-maintained.

And just because of this fragmentation, and you need to cover something that we call the matrix, which is a split by platform, devices, and OSes, you should use the cloud computing power to be able to perform parallel workflows and parallel runs. It doesn't apply just for the integration. It also applies for the delivery because in mobile you actually set up for distributing and building different build flavors.

So build flavor could be associated, let's say, with the classical development environment flavors that are out there-- for example, testing, staging, development, and production. But they map slightly, let's say, differently. It doesn't mean that it's actually just the back-end environment that changes there. There are, let's say, other characteristics that change between these environments, including the way the build is made and optimized for delivery.

In terms of distribution, let's say you obviously need to get the build in the hands of your stakeholders as soon as possible. And there are a lot of stakeholders and distribution channels, from testers to internally testing to alpha or beta testing your features or to the actual build distribution. And now since we kind of slightly reached the part where we actually need to do the deployment in the stores, the challenge here is that pretty often just because it can't be a continuous deployment at this time just because we still have the Apple Store and Google Play submission and review process being performed not instantly, this brings some additional challenges. Therefore, in mobile, most of the times the deployment is left as a business decision. And it's usually delayed because of the store approval.

At the same time, by using different technologies, there are some sort of ways where you can actually provide over-the-air updates on the apps that you have released in the stores without needing to pass again through the approval. But those are, let's say, more corner cases to do-- or technology-specific to do over-the-air updates or to do on-demand modules. Let's say for the majority of apps out there it's still done through the store approval.

Scott: I think that brings up the notorious server-driven UI question. There seems to be a lot of different opinions on whether that's a good idea or not. I know I've met a lot of mobile developers who say those software engineering things, test-driven development, and continuous delivery doesn't apply to mobile because we can't do that. Presumably, you don't agree with that point of view. Do you think we should still strive for continuous integration, continuous delivery with a mobile application?

Alexandra: Oh, yeah. Absolutely. For continuous integration, I think the state of the industry with cloud pipelines, the ability to target multiple, both physical and virtual devices is a thing of the current state. So that's definitely possible in terms of integration. In terms of delivery, I think that's also a state that we should try to achieve. However, most, let's say, development teams reach a cadence here. They don't do instant delivery of builds in the hands of the testers or other stakeholders.

They usually use a cadence. And I think it's mostly related to the fact that most of the CI/CD pipelines are still a bit restrictive or prohibitive from a cost perspective. They do offer plans that are, let's say, a per-minute usage on machine. And some companies try to optimize that, or they are per, let's say, a limited amount of compute time or hours. But it's very possible to do close to continuous delivery.

For the continuous deployment, things are a bit different there just because of a store approval. React Native, for example, already has the over-the-air update that passes the store approval. On-demand modules on Android are also close enough to this. And also, most of the apps have the built-in ability to-- are developed with the built-in ability to be able to do A/B testing as well as UI driven by the API.

But in order to perform UI driven by the API, you'll need to have, let's say, a base of, let's say, for example, a design token that will drive through the API the look and feel of the app.

Scott: I thought one of the things you said was interesting, that deployment should be a business decision. And that's how I've always seen continuous delivery — continuous delivery doesn't necessarily mean you're pushing every single change directly through to production, it means that the business can decide to deploy whenever they want. It's always in a state that's ready to deploy, and I think that point gets lost sometimes.

Birgitta: So I think Andres you talked a lot about —I think you used the word "specialized," "specialization" a lot and talked about the specific skills you need. And Alexandra, you also talked just about the specific challenges. So definitely there's like a separate skill set, let's say. Right? And I think this is often — Scott, you were talking about organizations struggling to scale this type of development, right? And then the first thing I think about is that we always try to have these cross-functional teams that do vertical slices, right? And I often see in organizations that got very far with this that the mobile team is the one that stays left over as the specialized team kind of, right? Andres and Alexandra, let's talk a bit more about this parallelization or finding modules that teams can work on in parallel, then the different specializations of Android and iOS. You know, how do you scale this organizationally?

Andres: It's a complicated question. The main question is how do you get all these developers to work on the same thing and, in particular, how can you use these two platforms that are so different from a perspective of development to be in a collaborative state. And this starts to sort of veer into the question of we have Android, we have iOS. Are we able to target both at the same time with the same technology? What should we be using to do that, right? And that's sort of a question that needs to sort of be answered before you start really building a larger scale in a larger system that you can — larger team that you can scale up, right? So--

Birgitta: The age-old dream of having one code base to deliver it all, right!? [CHUCKLES]

Andres: That's exactly right. Yeah, yeah. So that's the question. There's been various approaches in submarine land slash Maui using C-sharp to target both to having teams that have parallel sets up to React Native and Flutter more recently. I think there's been a very, very collective set of opinions here as to what should be the current state or the right state. 

And you'll find that in some places they're delivering things in one way. And you may go across town, and you might find another place that is delivering with completely different standards and just doing pretty well anyway. So I don't think we've reached a standardization point yet. But I will want to point out that in my heart I've always believed that being close to the platform and being able to do native delivery well has brought for many of the clients that I work with an edge, and extra value, and a competitive edge at the end of the day.

Alexandra: Yeah, absolutely. I don't think we are in a state and we are not going to be in the next years also, where there will be a one size that fits all in doing mobile development. And we are at the state where, let's say, a native as subject matter expert it's still required. Even let's say the specific software apps allows you to have a common business logic, for example, written in a common language. You will still need required native for more specialized integration, especially if you have performance in mind or interacting with hardware. Also, let's say the mobile space is more dynamic than everything else.

There have been a lot of changes throughout the years. And I think that one thing that helps is that as an engineer you should not be attached to a particular toolset or programming languages. Over the years, we have seen going from Objective-C to Swift, from Java to Kotlin, or even with React Native going from JavaScript to tie straight now to Dart. So it's not about the particular framework or the particular technology. I think it's more about the architecture and the team topology that results from it.

Scott: That implies you've got some kind of modularity, right? Does that work in a mobile application? How do you achieve that kind of modularity where you can have multiple teams working on the same application without stepping on each other's toes?

Alexandra: There are actually two, let's say, main categories of architectures which fit mobile really well. One of them is modular, and the other one is event-driven or reactive. Now focusing a bit on the modular part, what usually works well in a modular architecture is to actually do a bit of, let's say, domain boundaries analysis, where you can structure your app into, let's say, a layered modular approach.

So have some sort of core specialized modules that can be shared through abstractions and dependency inversion into the application, as well as specialized-by-feature modules that are independent and mapped to business features. This will give teams the autonomy as well as the ownership of specific features from, let's say, the inception of a feature, the design of the feature, as well as owning its full development lifecycle.

And let's yes, modularity comes with a lot of challenges, especially on the integration side. That's also required. But if we apply, let's say, solid principle like single responsibility and dependency inversion, you should be able to elevate the integration points as well.

Andres: You know, I agree 100%, particularly when we're talking about good old-fashioned engineering. I think that solves so many problems that we tend to look for buzzwords sometimes, where just using the tried-and-true tested tools is something that has a lot of value. And I would add that, on top of having that approach, let's keep in mind that the modularity or the architecture of the mobile apps don't happen in a vacuum. The shape of your team will also determine part of the architecture.

So for instance, there's the old idea that using Conway's law, if you try to set up three teams to work on a compiler, you're likely to end up with a three-stage compiler. Right? So the idea that you can do something technical that will end up in a better model utilization, that's true. But there's limits because the way that you set up your teams will also determine the overall architecture modularity of your applications; and mobile is not an exception of that.

Scott: So how would you organize your teams in an ideal world? Would you have them organized along business domain boundaries or functional boundaries? Or does it depend?

Andres: Generally, I think for people to perform well they need to feel like they're bringing value to the table. And by aligning teams with specific value creation activities, we have better outcomes, clearer boundaries. And in the end, because the technology comes from people, the way that they interact, and the way that they work together, I think it's more important to take it from the human and the business side and let that sort of imbibe the technology rather than the other way around, right?

Birgitta: So when you say value-aligned structures, then immediately, of course, I have to think about team typologies, the streamline teams, platform teams, and stuff like that. So have you also seen this in terms of mobile platform in this whole — I guess when the scale gets bigger, maybe having like one team that kind of builds a kind of platform to enable other teams to build the mobile features on top of it? Like, what does a platform look like in the mobile world?

Andres: It depends on the scale of the business, but generally, yes. I've seen platform teams one that may specialize in iOS and another one in Android. They work together to serve the rest of the development team and say, hey, these are the set of tools that we prepared for you and then in a very really servant manner to the rest.

And they try and say by observing the latest-- and like Alexandra was saying before, because things move so quickly in the world of mobile, they need to observe the latest changes in Apple and Google, and then take those and come into the teams, and really bring those in. But really the special cultures that are part of the periphery of Apple and Google as development ecosystems, that should also be part of this conversation. It's not just a platform from a technical perspective. But there's also very specific cultural ideas and cultural memes even that are repeated through the organization that should be part of it.

Alexandra: Yeah, absolutely. To add to this, I have seen platform teams especially in setups where organizations have not one app distributed on the two platforms but actually multiple branded apps. So in order to do this, let's say engineers work with a more SDK-mindset. And the platform side of things helps here because it will allow you to develop modules that can be used across multiple applications without having to deal with code duplication or too much of a fragmentation.

And that's all having, let's say, an SDK-mindset in building these modules. Also works when you have to distribute them publicly or even for internal distribution for different apps. It would also help too with versioning and apps being at, let's say, different maturity stages, different release cycles.

Scott: You mentioned the succession of technologies, how we go from one programming language to another. And they kind of go in trends. And one of the things I've seen is all of that in one application. I think that people tend to keep these applications going. And they develop them over years, and the technology changes. And they also maybe didn't put a lot of thought into the architecture when they first started building it. And so I wonder — and I get asked a lot of times where do we start? How do we-- can we salvage this? Like, how would you approach a big monolithic application that was constraining how parallel the development could be on it?

Andres: So that's a very, very common scenario. And companies usually come with multidimensional problems, where they say we have this much money. Invest it into this application. We don't know what to do next because it's using these old technologies that are no longer supported. They might be targeting older versions of mobile operating systems. They don't know how to move forward. And they're sort of stuck, and it's a difficult situation. But what I do in those cases is to really have a team that is really, really good at integration.

When your team is fantastic at integration, they can solve these complicated problems and slowly move ahead by untangling the cross-requirements of, say, if there's a portion of the app that depends on a specific old version of a library or something like that. They can very carefully untangle that and use, say, native integration to make calling back and forth. And that way you can slowly move forward but very, very carefully. These are very, very common situations. Yeah.

Scott: By integration, do you mean integration with the underlying platform, with the native platforms?

Yes, absolutely. Absolutely. I think also if you can find programmers who are really good at low-level programming, that can really help you. If they understand the underlying platform and they understand how everything fits together, they can slowly untangle the web.

Alexandra: Yeah, absolutely. And also, let's say the incremental approach is very important here, incremental for re-architecturing as well as rewriting because you want to keep the business continuity since at that point the application is already live. And users do want subsequent releases. So it should not be one shot replace all but more of an incremental approach.

On the other hand, I think there are some similarities with, let's say, the six errors of cloud migration, where probably retire is the one that doesn't make too much sense. By retiring here-- but retiring here could also apply to we are going to switch incrementally to a totally new application. But it's going to be seamless and frictionless for the users.

Let's say the other re-hosting or re-platforming could simply transform into let's use a different technology under the hood. Let's use a different delivery or deployment process under the hood.

Scott: That's interesting, yeah. Applying some of the concepts of cloud migration over to mobile, that's an interesting analogy. One of the things that I've heard about a lot lately is super-apps — and that’s related to this idea of scaling. Are you seeing that? Do we see apps that encompass a whole bunch of different kinds of functionality in a single app?

Alexandra: Yeah, definitely. There are applications that have a lot of products in their souls. So it's not just one application product. They have a lot of integration with not only backend services but, let's say, third-party hardware devices. And apart from just being a super-app in itself, usually, companies also have other channels. And they strive to provide, let's say, an omnichannel throughout, let's say, all of their estate of applications. And indeed, there are a lot of, let's say, growing pains with this.

But I guess some of the, let's say, techniques that we discussed before, such as being modular, finding, let's say, the right split in terms of architecture, code bases, and themes, will also help with super-apps. Also, what usually helps with super-apps, just because it grows so much, it's also move from a mono-repo to multiple repos. This would break dependencies even further and allow for more autonomy. And also it will allow for, let's say, different products within your app to have their own independent release cycles.

Scott: Yeah. That mono-repo thing, that's another controversial topic, I think, in here. I'm not sure. I see it more with mobile apps, where they throw everything into one big repo.

Birgitta: Which makes sense because it's one application that gets released, right? So I think it makes sense in the first step. But it's good to hear that as it gets to a certain scale it's also possible to split that up into multiple parts. Yeah.

Alexandra: Yes, definitely. Splitting it will help with, let's say, easier versioning, decoupling the release cycles. And let's face it. It will also help with the developer experience. Smaller codebases which are independent will break, let's say, the team dependencies. And they will eventually actually improve the, let's say, experience of the developer as well, not only bring benefits to the app itself.

Scott: I suppose that means you need to be able to test the modules independently and distribute them as binaries. Is that right when you take that approach?

Alexandra: Yes. So with splitting either by modules or in the multiple repository, usually, the modules are then delivered to different teams for integration as pre-compiled, which actually also helps with the build times and delivery times. On the other hand, another benefit would be that you can actually, one, associate another test app with the module so that you can be able to test them more independently and only delivered when you are actually green.

Scott: So where is all this going? Where do you think it's heading? What do you see as the trends going forward into the future for mobile development?

Andres: That's a really good question. I think some of the things that Alexandra mentioned are definitely great data points. We're looking at super-apps. We're talking about more integration. I really think that in a few years we'll be seeing apps that are very, very integrated but also that are able to do much more in one go.

So instead of us having to do little pieces of a task, if I want to have a holiday, I should be able to book that holiday and prepare it as one discrete task rather than break down buying the tickets, then booking the car, then doing little pieces. And I think you're going to be seeing apps that are highly integrated with what you do as a person and with different service, many different services and platforms, and will take care of bigger chunks of work, allowing us to be more efficient in one go. As an example of maybe a platform that will do something like this is here in Australia Thoughtworks we are working to build a digital per-certificate app.

This is for the government of New South Wales. And it's got a vision for it to become a platform or a framework for many other applications that would sort of reduce friction for a lot of people. And I think that's sort of a forward-looking idea of where we could be going in terms of how well the government could be integrating into our daily lives through our apps and allowing us to do those larger pieces of work that I was talking about before.

Alexandra: Yeah, definitely. I also think that the trend will continue, where more companies will go mobile first or even mobile-only when the launching their projects. And I also think that in the near future we will be able to do more with mobile with the, let's say, increasing processing power as well as being able to tackle more directly into the operating system or hardware accelerators. And hopefully, some of, let's say, the current pains will be alleviated. For example, I can see AI doing the review process automatically. Therefore, perhaps we'll have the actual continuous deployment and delivery happening.

Scott: OK. Thanks. That's another podcast! [LAUGHS] Every conversation gets to generative AI at some point for me these days! So it's interesting. Thanks a lot, everybody. I appreciate your taking the time to do this. And it's been really interesting for me. So see you next time.

Birgitta: Yeah. Thanks.

Andres: Thank you.

Alexandra: Thank you for having us.

[MUSIC PLAYING]