Software Architecture: The Hard Parts

Full transcript

Rebecca Parsons: Hello, everyone. Welcome to another edition of the Thoughtworks Technology podcast. My name is Rebecca Parsons, the Chief Technology Officer for Thoughtworks. I'm here with one of my co-hosts on the podcast, although she's here today as a guest, Zhamak Dehghani.

Zhamak Dehghani: It's great to be here, Rebecca.

Rebecca: We're also joined by someone who's has also been on this podcast before, Pramod Saladage.

Pramod  Sadalage: Thank you, Rebecca. Nice to be here with you, Zhamak.

Rebecca: Thank you both for joining us. What we'd like to talk about today? We foreshadowed this in the episode when we had Mark Richards and Neal Ford talk about their latest book, Software Architecture: the Hard Parts because both Zhamak and Pramod collaborated with Mark and Neal on this book to add aspects of data architecture. We wanted to get them on this podcast as well to talk about their experiences with that. Let's start with the mechanics almost.

Mark and Neal have been collaborating with each other for a long time. I do know, Pramod, you did collaborate with us on the building evolutionary architectures book, but how did this collaboration come about? How did you end up being co-authors on this book?

Pramod: You want to go first, Zhamak?

Zhamak: Sure. Neal and Mark have been working on this material for a long time. they had a wealth of information, to share off the back of their book that they had written on foundational concepts in architecture. Neal invited me to join because, at the time, I was working on data mesh as a paradigm shift in data architecture. We wanted to really extend the conversation around architecture to data, and particularly, usage of data in analytics and ML. I did not take a moment to even think about it. I said yes, immediately, because I knew it would be an amazing learning experience to work with Neal and Mark to contribute their book.

Pramod: My story is something similar too. I have been, like you said, Rebecca, working with Neal on some of these concepts of like the intersection of software architecture and data architecture, especially around the concept of how you would evolve some of these things. The contributions in the evolutionary architecture book, as well as some speaking engagements and some O'Reilly software conferences, and things like that. Neal asked me if you'd like to contribute to this book, and just like Zhamak, I jumped on the idea, and here we are, probably after a year of banging on the keyboard.

Zhamak: I think there is a little grain of truth that I think is the sub-context of actually this book or this work that, for a long time, our specialties have been divided. A bunch of us were very focused on the computational architecture and operational. It feels like Pramod had great experience on the data aspects of it more than I. I think this was a great opportunity, even from the skill set and experience and a specialty, bring folks like Pramod who had that data focus, and perhaps, Neal with a computational focus, and it was necessary. I think it was necessary for Pramod to join us.

Rebecca: How do you see these ideas around data architecture fitting into these broader concepts of architecture? I know, we've been evolving our thinking about what constitutes architecture for quite some time, but did it feel like, "Okay, we've got all of this computational stuff. Let's pull the data stuff on the side," or did it actually feel like it fit into to the book as the two of you were working with the two of them?

Pramod: Yes. In some ways, I would say it fit very well and the reasons being, like any trade-off analysis, you do, there are probably I would say, 10% to 15% of that that doesn't involve any data, but majority of it is involved. You are talking about like, "Should I take eventual consistency or not?" That's not just a software decision. It's all about data. Similarly, there are other discussions about coupling versus cohesion or versus things like, should I take on distributed transactions or not?

Bunch of these all involve the data, and whenever you're making decisions about the architecture, underlying theme that you are trying to drive at is, how do I make data available? How do I keep the data consistent or how do I keep all these pieces together so that I don't have to take on other accidental complexities because of my decisions? Some of these, ultimately if you keep digging down further and further, they affect data and how the data is available. Either to the applications or to the analytics platforms so that you have to involve the aspect of data into those decisions.

On the surface, it may seem I'm making a software decision, but if you dig deeper, you can see that data is a big driver into what decisions you arrive at.

Zhamak: I think on the analytical side, on the operational data storage and how do I model my data for my services as Pramod said, it's actually a bit closer to the decisions that you make around the design of your applications, is part of that. From the analytical perspective, it's often been a very disconnected set of decisions and trade-ups. One thing about this book that actually brought the analytical decisions closer to the operational decisions, was the narrative that the book explains on behalf of this organization, the hypothetical organization, Sysops Squad that is. It's an organization that provides technical staff to support staff for people that bought electronics for example.

If you ground the decisions that you have to make at an organizational level in a real or even a hypothetical example, you will see that your operational structure, how you design your services and your applications, easily gets extended to, how do I now expose data from these applications so that I can now make intelligent decisions for the Sysops Squad? In fact, we have a narrative that follows through the book that puts the reader in the shoes of decision-makers in this organization. Initially, the decisions around how do we build our applications? How do we allocate the data or organize the data for those applications?

Very quickly as that organization evolves, they start thinking about, how can I optimize my workforce based on the knowledge that I have in terms of the demand? How do I upscale my workforce? How how do I make intelligent decisions around the routing of this? Support requests are coming. The moment you start extending those operational capabilities to intelligent or intelligently augmented decisions, then we have to bring the conversation of analytics as part of a bigger picture of architecture conversation. It can no longer be a separate conversation. I think that was a wonderful thread through the book to bring all of these decisions together. Same as what happens in reality in real organizations.

Rebecca: It is interesting to me how in the past there was that hard separation. This is the transactional data store. This is the operational data store. This is where the work happens, and then we'll toss everything over. That's where the analytics goes on, and there was such a separation. I like the way you phrase that, Zhamak, and thinking about as we want to become more of a data-driven business how we have to start dragging that analytics back into it because those workflows are now requiring access to information and to that analysis of the data. It used to be someone else's problems.

Pramod: I think the other aspect of this is also even operational systems nowadays need analytical data. If recommendation engines, for example, they're right there on the website along with the transactional systems but they need analytical data. You can't rely on analytical data. That's a week old because that's when the data ran or you can't rely on them because of whatever other circumstances. It's like now we have to think more about, how can I get analytical data faster? How can I get analytical data in a shape that something more operational can use it? You have to think a little bit different about where the systems fit.

Rebecca: Let's step a little bit further back in history because for a very long time, well, first, for a very long time, architecture was simple. You had a mainframe. Then oh it's now client-server and we have a much more sophisticated understanding of architecture and the list of the different kinds of architecture; security architectures or network architectures, software architectures, integration architecture, all of those different kinds of architecture. For a very long time, people just talked about data. You never really heard data architecture. What do you think was behind this evolution of thinking as it's not just the data but we're going to elevate our thinking about data to this term of data architecture?

It's not like we didn't have these problems, we had to think about where data was and we had to think about issues of data locality. What do you think has happened that has allowed us to get this more sophisticated way of viewing data?

Pramod: I would say multiple trends, a bunch of trends back like maybe 20, 25 years ago, like you're saying, clients servers, most of the data was in one database, and you just connected to that and just used it. Like Zhamak was saying, you take the data from there and pump it somewhere, ETL or ELT, or whatever, and you have a data warehouse, very distinct set of requirements and distinct needs to be met.

The more cloud adoption or maybe even the whole concept of using data for more purposes, capturing more amounts of data, capturing data from different sources, like the speed at which the data came at you, and the various sources where it came from and all of that stuff, exploded the ways in which data can be used, should be used, and the way value can be captured from that data. All of these forces are now giving more importance on how we can use the data.

That's why I think now data is coming more in the center, of course, like cloud adoption, cloud databases, all the technologies are also helping us solve that problem. That's what is, I think, making data be a more prominent actor in this software architecture landing.

Zhamak: I agree with that. I think there is another element here which is the complexity of our organizations adjacent to our aspirations and all the diversity of the use cases of the data that we have. Increase the scale of organizations, just simple, I guess, physical factors, entropy, we have more and more business units within a lot of organizations. There's a lot more interactions and interconnectivity between these business units. The organizations have a continuous drive for growth, which brings complexity to the picture.

At the same time, now we have the volume of the data that we need and we have the computational power to really think big and aspirationally in terms of how we can use the data in variety of machine learning models and in optimization of our workforce and processes and ML-driven features. If you look at that complexity and then look at what we had built architecturally, there is a mismatch here.

Architecturally, again, we haven't really paid much attention, at least on the ML analytical space. I think on operational space, we have had advances. In the analytical space, it was just good enough to run those ETLs, get the data out with blood and sweat and tears, convert them through the pipelines, and put them under some sort of a schema. It doesn't matter how long it takes, it doesn't matter how different the data gets modeled from the reality of the facts, it's good enough for reports.

Given the complexity and aspirations and proliferation of use cases and proliferation of sources of data, that no longer works. If we step back and think about what is architecture? What's the role of an architect? Neal has this very-- I know there are many definitions of architecture and I love many of them, like the framing that Martin proposes is, decision about hard things that are hard to change.

Neal has also this wonderful description that architecture is about-- lots of architecture decisions are around how we decouple systems so we can get some sort of an optimization around scale, around scale, around speed, and then how do we integrate them together? In fact, the book has been structured around patterns of architecture that allows decomposition and patterns of architecture that allows reintegration. The moment we think about now this big data that we thought it was step problem, side problem and in the side of the organization we will have a warehouse, we'll have a leak. Now we have to think about how do we decompose it so that we can respond to that scale and complexity and multiple demands of the organization.

Then how do we connect this data back so that we can get this high-level intelligence and insights out of reconnecting the data back together? That's exactly the problem that architectural software or architecture at least has tried to solve. It was really interesting to me to see that early 2020, InfoQ put for example data mesh as an architectural trend on their architectural trend publication that they publish once a year. When they did that, they told me this is the first time that on this publication we have included something around data architecture. It's the first time that really the conversation around how do we make architectural decisions, around at least the analytical data, have entered the conversation.

Rebecca: I do wonder, picking up on that idea that you were talking about when you look historically, particularly in the pre NoSQL database, you had data warehouse-style architectures that were handled by the analytics folks over there to the side. You had relational databases that was handled by the database administrators. There was just one thing there. Now, in part because of the advent of NoSQL databases, but I also with this moving in of the analytics and some of the problems with distributed data and scalability and the proliferation of data sources, it isn't any longer any one role's problem to think about this.

There are different concerns and in many of the architectural conversations I've had with Neal and Mark, the word trade-off just comes up all the time. When you've got trade-offs across different problem domains or different technology domains, those trade-offs become even more complicated. That to me is what architecture is about, is, you've got a problem you are trying to solve, you have a series of constraints that exist and you've got to make trade-offs amongst what you're trying to achieve on the basis of these constraints.

I think just that overall complexity of the data landscape where you can't deal with complexity unless you have some framework for thinking about it and that's what architecture is all about. Fascinating, that's a question I've been pondering quite a bit is what does it take for something to be able to be called architecture?

Zhamak: I really like actually this trend that's happening. I feel like when we think about evolution of technology, sometimes a cool new tech comes along and that technology drives architectural behavior. In many cases, an architectural paradigm comes along and it transcends the existing technology and then the technology needs to catch up and adapt to it. I saw this model with the microservices back in the late 2000s like 2011, I think, we started the conversation by 2012, 2013. At least I was knee-deep in projects building microservice.

I remember there was a friction with using the technology we had in this fine-grain decomposed architecture. Over the course of the last decade, we saw containerization, we saw the orchestration tools. We saw all of this technology that had to come along to really make that paradigm feel native to the developers. I really hope by immersing more of the architectural thinking into the data landscape, we can create these meta frameworks that try to solve a real-world problem at that moment in time that perhaps Transcend Technology and then the technology comes along to fit in that paradigm. I'm excited about consequence of talking about data architecture.

Rebecca: Pramod, tell me a little bit about your contribution to the book. How did you decide what scope you want to address and what's the key takeaways are from your perspective?

Pramod: Yes, like Zhamak was saying, the two patterns we try to access, lay out everything under is the patterns for decomposing and the patterns for bringing things back together or integrating. Like, what are the forces that are driving you to decompose things or to bring things back together? We go through a bunch of patterns like that like for example, scaling. If you have to scale certain things, then maybe a monolith architecture may not work for you. Then what are the forces? Is it the amount of connections of the sheer size of throughput or what are those forces that force you to decompose certain things?

The inverse of that is, if you're ready to decompose, maybe there are forces that are wanting to put things back together. One classic example of this is distributed transactions. For example, if it's too hard and if you can probably put things in one service that can take care of functionality that is common, maybe that's a driving force to put things back together, maybe make the domain bigger or the service bigger and that kind of stuff.

Once we get up to this point, like then we said, "Maybe we should give some concrete examples on like, if you are going to break things apart, what are the ways in which you can break things apart? So, we take the reader through a journey of like, first of all, defining what a data domain is. From a domain-driven development concept, we have come to understand some concepts inside the business have this functionality that can be a domain and they do certain functionalities, and how does that relate? How does that domain in the business and domain in software architecture relate to a data domain?

What is the data domain? What are the pieces of a data domain? How do you define adore data domain and things like that, and give you an example of, again, using the Sysops Squad as a practical example that we take throughout the book, it's the same example, same concept, and same-- One funny thing I thought was really good that we did and was the concept of same developer. Even the people, people names like developer name, data architect name, float throughout the book. If you hear Dana, Dana is a data architect, and she's thinking about all these terms. Starting from chapter one all the way to the last chapter, you can hear Dana's voice as a data architect, giving her input and that kind of stuff.

Dana starts thinking about, what is the data domain, how I can separate, and that kind of stuff. Once we take Dana through the thinking of what is a data domain, we have laid out the concept of breaking apart a monolith. There's a lot of talk about how do we break apart a monolith and how it's hard and how it's not easy and that kind of stuff. We lay out a five-step process on which people can go about breaking apart a monolith, like of course applications have to be live when we do this. Bringing in some concepts from the database refactoring book into this concept of how you can break things and then lay out a five-step process for that.

At that point, we said, okay, this is good enough to talk to people about how to break apart monolith but we thought there is a topic that we have not dealt a lot. Like nowadays, the amount of choice you have in terms of what technology to pick for a database or a data storage at a very high level is immense. It is confusing, it is on which all trade-offs are based. Like, do I pick a key-value versus do I pick like a graph or do I pick a cloud-native database like Redshift or Aurora or something, or do I pick snowflake or, do I pick CockroachDB? Those choices are immense.

We broke apart these choices into, I would say, eight or seven distinct buckets. Each bucket will got certain rating and in The Fundamentals of Software Architecture book, Neal and Mark had come up with this rating system for certain things. Mark especially encouraged me to think about rating system for database choice and we ended up doing that for these seven or eight different big buckets of things and giving them ratings around software access. Like, would you use this for consistency purposes? Would you use this for throughput purposes? Would you use this for ease of developer? Sometimes that's a choice. How easily can I use this?

The other choice would be around tool support, if you're using some SQL language, does the database support that particular tool choice and things like that. Giving you a star rating system on those databases so that it helps the reader to make a choice. Now, of course, every reader is in a different context, but given that context and given this star rating, maybe make it easier for them to make a choice on what technology to use and what database storage mechanism to use in their architecture. I think I focused more on the operational side of the data. Let the Zhamak focus on the analytics side of the data.

Rebecca: Okay, so, Zhamak, what would you want readers to come away from in the sections? I'm going to take a wild guess and say it has something to do with data mesh?

Zhamak: It does. It does. In fact, I contributed to this book while I'm deep in the data mesh space on writing a book on data mesh. I think it was a little bit challenging for me to step back and stay away from my emotional state. From the evangelist of data mesh to really sink in with an unbiased view, what are the trade-offs between the different analytical architectures because this book was about that, was about as promotes the different, trade-off, a framework to give users to make decisions for themselves.

I feel my job was a bit easier than promoted, even though that's in the operational space there's so much. Like there's so many complex decisions to make around the caching and the replication and the mode of access. Those exist in the analytical space as well, but from the bigger picture architecture perspective, there are not that many different architectural styles. In my section, I think what I would like readers to take away, first put themselves in the shoes of the Sysops Squad and see why they are actually making decisions around their analytical systems. What's driving that, what business decisions are driving that.

We have that narrative in place around how data can optimize the business of Sysop Squad and the experience of the customers, and then look at the moment there's three macro architectures that exist: data warehousing, data lake, and data mesh. I know there are some technologies in the middle, such as lake house, the lake want to look like a warehouse and warehouse wants to look like the lake, but at the macro level really, those are these three big decisions or big options that you have.

Then look at the advantages and disadvantages of each of those. We have that laid out in the book. If you're starting and you're not a very large complex organization, a very centralized, consolidated approach like your warehouse may be okay, but know that would limit the functionality that you can run on that warehouse. Know that there's going to be extreme partitioning between the actual domain knowledge of data and where the data gets modeled? The domain knowledge is really within the tech and line business domains, but now you're moving the data away from that to a data warehouse and the data warehouse team.

If that's not good enough, have a look at the advantages and disadvantages with Lake, that Lake perhaps it gives you a bit more flexibility because it doesn't require that rigid schema modeling of a warehouse, that really tightly coupled like Snowflake's schema with bring some rigidity and slow you down. It doesn't have, that this is less structured, but it's also more prone to going stale, more prone to have difficulty understanding the domain context within this data that we dumped into the lake.

If these are not good enough, then move to data mesh and look at what data mesh advantages and disadvantages are and what we try to do. I think I've talked about that enough but really it's about getting that agility and the speed that you want and bringing data usage and data consumption as close to those operational teams and as close to the operational databases, but yet realize the respect the differences between the two.

Data mesh really lends itself to modern architecture principles and practices around domain modeling and around loose coupling. It allows you to really have that holistic architectural conversation, not just a data conversation in an isolated fashion. In the book, then we went through a little bit deeper on the architectural aspects of the data mesh. We introduced this concept of data quantum, which I have shamelessly borrowed and stole from you, Rebecca, in fact. Maybe you want to talk to that a little bit.

I love this concept because when I recall the definition that you have in the book, you wrote evolutionary architecture is the smallest piece of your architecture that structurally has all of the elements that it needs to autonomously do its job. In the case of data mesh, data quantum is the competition and the data and the policy, all of the structural components that makes a domain-oriented data autonomously, useful, alive, timely, respecting privacy, all of that.

It's a new concept and I know the word can be a little bit alienating. People go, "What do you mean by quantum?" I love the concept and I took it from evolution architecture. We go a little bit deeper on what is this and how does this fit this data quantum into the bigger architecture.

Rebecca: Given the fact that you both are on this call and we've talked a lot about how notions of data have evolved over time and such, and yet, it still does feel in particular, Zhamak, towards the end of what you were talking about, that there still is a distinction between the domain of analytics at using domain in a different way, but that you really should think about analytic computing and operational computing.

That they are still different even though as Pramod mentioned earlier, many of our operational systems now require analytical data. Do either of you foresee a time when this hard distinction between operational systems and analytical systems are just going to go away, or do you feel there's something fundamental about that divide?

Zhamak: I can have a go at this. It goes back to that question of what is architecture. How do we decouple responsibilities? We still have that delineation of responsibility and then how do we integrate these pieces so there is this continuous feedback? I think maybe there will be a point in future that our technologies are just so universal that it doesn't matter that I'm running a transactional system that requires a certain response time, at certain workload, a lot of reads and writes at the same time, and the teams that are maintaining that, really caring about the current states of that system.

Maybe there is a time that operational systems, we can come up with technology that universally satisfy those. At this moment in time, I know your question is really about future, but if I look at just what I can see ahead of us for a little bit longer, those still have certain characteristics that are quite unique to them. As an example, the relationship between compute and storage, the relationship between code and data in the operational world, is that in a way that you write the code, you write your imperative code. As part of running that code, you’re generating states that then you persist. That the state is really here to be supportive of that computation.

In the analytical world, when that data gets changed, modified, and transformed to be used, the relationship between competition and code changes. In the analytical for in machine learning model, in fact, is the data that is feeding the computation and making decisions. We are finding patterns and retraining our machine learning models. The relationship is different. The nature of the access is different. Why they need to be integrated, tightly integrated, feedback into each other, operational data feeding into analytics, feeding into this temporal view and aggregate view of the data and the intelligent feeding back to the intelligent API's within the operations.

As far as I can see right now, I think they have a delineation of responsibility. That requires us to think about them as slightly different but yet integrated.

Pramod: Yes. I would agree with that. I also add a little bit around even within the analytical side or the operational side, I would say 20% to 30% is in that other circle, If you draw a Venn diagram of operational systems and analytical systems, there is some commonality. That commonality is where the integration happens tighter. The other things that don't intersect at all, that's where we tend to think of them as separate things like this deep analysis, our data scientists digging through the analytical platform to find patterns and things like that are on the operational side trying to optimize the system to serve, give more throughput or serve more transactions and things like that. Those two don't have anything to do with each other in some ways.

At the same time, the integration batch of data from analytical side or from operational batch to like, how can I get operational data faster to the analytical platform is a question that people are asking now more than they used to probably five to 10 years back. People would just say, how fast can I run this batch? Two times a day, three times a day? Now, people are asking, "How real-time can I get this data?" They are no longer talking about batch and that's a change in thinking on like how much integration that is needed into these two sides of the organization or of the architecture as such.

Zhamak: I think one way to think about how do I separate and how do I integrate is really think about this domain concept of domain-oriented design. Organizationally, we have decoupled ourselves over the last decade around the business and business domains and aligned our technology and services, and applications with the business. What we can do is to say, okay, so we have all of these wonderful different domains. We have the order management. If you're in retail, we have e-commerce, we have customer management and they have a set of services and technologies to serve the business outcomes.

Let's extend that to now analytical data-sharing and analytical data-capture and analytical use cases within that domain so that integration and that feedback loop becomes quite tight and rapid because we have now this domain alignment, and of course, we have then interdependency with contracts across the domains but we move away from this functional siloing and decomposition, analytical and operational to a world of decomposition around, really, domain outcomes and domain capabilities. Within the domain if zoom in, yes, there will be some analytical pieces and there will be some operational pieces.

Rebecca: One last question for each of you, we now have this thing called data architecture. What would I have to do to be a data architect? What advice would you give to people? How should they pursue thinking about some of these questions of data architecture?

Pramod: I think data architecture in some ways is like having empathy for the users of data, thinking about how this would be used. Don't think of it as a thing to control or a thing to design for. It's more about how is your data going to be used and where it is going to be used and then how well can you provide the data back into the organization or into the other use cases, whatever that may be. Then go down from there in terms of tools, technologies, people, process, all of those stuff come into play.

How can you organize your team so that the data architecture is working good for them, is being more productive for the developers, and things like that? How can we organize the data in such a way that it is productive, again, for the business users and things like that. I generally focus on that side so that the architecture is providing for the end-users in such a way that it is useful. It is making them productive. The end-users of an architecture are of course the business people, but they're developers. There are other stakeholders, there are many different kinds of stakeholders.

I generally tend to also work with this software. We are not necessarily fighting with the software people. Data architecture is part of software architecture. We just care more about the data and they care more about the software, so how do you come together to care about the architecture? If you look at it that way, then you can work within the team of software architecture to deliver whatever architecture the company needs or the organization needs.

Zhamak: I really want to emphasize that last point. I think data architecture or data architecture decisions should not be isolated decisions. They have to be part of a bigger picture architecture and that bigger picture architecture is both around applications, integration of their data into ML and analytics, and integration of that back into the application. Application system architecture, analytics architecture is all just one big architecture decision.

Of course, when you go deeper, you might need to make specialized decisions but a lot of the decisions are about the integration, the harmonious kind of integration of these independent pieces that need to work together. Maybe we're all talking about architecture and then yes, there are some specialized fills in the middle, but at the end of the day, we're making arch decisions.

Rebecca: Excellent. Thank you, again, Pramod, we've been cycling you through a few of these podcasts lately, and thank you, Zhamak, for joining as a guest, as opposed to a co-host on the Thoughtworks Technology podcast. I hope people found this discussion useful. I personally found it fascinating, so thank you both Zhamak and Pramod.

Pramod: Thank you, Rebecca.

Zhamak: Thank you, Pramod and Rebecca. It was wonderful.

Rebecca Parsons: Hello everyone. My name is Rebecca Parsons, and I'm one of your recurring co-hosts on the Thoughtworks Technology Podcast. I'm here today with Neal Ford, who was not here as a co-host but as a guest, as well as Mark Richards, we have a podcast from a while ago where I spoke to the two of them about their book, The Fundamentals of Software Architecture. They're here today to talk to us about their forthcoming book on Architecture: The Hard Parts.

Neal Ford: Wow, the guest chairs in the studio are much more comfortable than the podcaster chairs, so this is really a treat. Thanks for having us.

Mark Richards: Thank you so much, Rebecca.

Rebecca: The first question that came to my mind when I heard Neal start talking about this book is first, the title, The Hard Parts. You just finished a really comprehensive book on the fundamentals of software architecture. What more could there be? What prompted you all to do yet another book talking about architecture, and why scope it to the hard parts?

Neal: It's a great question, and a great obvious first question. As we were writing the fundamentals book we kept running into these problems that we would start diving into and then realized, "Oh no, this is a rabbit hole. This is going to take forever to get into all of the different possibilities and trade-offs in this." We started putting those aside and saying we'll do that in some future book or something. Mark is actually credited with coming up with this name because he said at one point, "Hey, I know what we should call the next book. It's Architecture: The Hard Parts," because it's all these difficult problems that we've set aside from the fundamentals book.

When we got the fundamentals book done, we started looking at this pile of difficult children that we had built up over time and tried to figure out, what is making these things so difficult? What is the thing that makes architecture hard? If we're going to call it the hard parts. We realized that all of these things that we're having such a hard time talking about in a simple, fundamental way is because there were so many trade-offs associated with them. It wasn't a simple linear decision path, it was you get to here and you have to make this trade-off decision, and here and make this trade-off decision.

That's when we realized that, oh this is not just a collection of things in distributed architectures. It's a book about trade-off analysis and that's the subtitle of the book, is Trade-Off Analysis for Distributed Architectures. That's where we thought we could add some actual insight into distributed architectures is, how do you do trade-off analysis in modern architectures? There are a lot of older techniques that don't work for various reasons, and a lot of teams struggle with this, particularly in microservices. There's some really fundamental decisions that you have to make that are nuanced and have a lot of trade-offs. We wanted to show people how to do that kind of trade-off analysis.

Mark: The other interesting thing, Rebecca, about this was we kept adding more and more information onto The Fundamentals of Software Architecture and that pretty quickly got to around 425 or close to 430 pages. We wanted to talk about all these other aspects of architecture in that book, but there just simply wasn't room. One of the things we realized for example, let's talk about distributed transactions, and it wouldn't really do justice just to spend maybe one or two pages on it.

This book really allowed us to take all of those things we weren't able to talk about in the fundamentals book and really talk about them, but more importantly go really deep into each of these topics, and so it's a such an excellent follow on book as next step to say, "Okay, now that I understand the fundamentals, let's go deeper." [chuckles]

Neal: In fact in the fundamentals book we had a chapter on microservices, which is hard enough because there are entire books written about microservices and software architectures and hard parts, in many ways is about just two aspects of microservice architecture. It is digging in a layer upon layer upon layer of these complex problems. It'd be hard to do in a single book.

Rebecca: Can you talk a little bit more about how the way you approach trade-offs has changed over over time? I can imagine an intellectual framework that said, "This trade-off analysis to a certain level of abstraction, is the same one that we have been dealing with for forever." How is it that this trade-off analysis really has evolved over time?

Neal: There were a lot of attempts to make formal trade-off analysis methods in architecture, ATAM and CBAM and a bunch of other- interestingly mostly 4 letter acronyms rather than 3 letter acronyms. I think because they're that much more complex than the 3 letter acronym things. [laughs] There were a lot of attempts to do that, but they were all very heavyweight, very bureaucratic, very time intensive and a lot of meetings for people that you could never actually really get to meet together, but the aspirational goal was--

The problem you have in modern architectures is not so much the trade-offs that you were trying to arrive at, which were these holistic enterprise architecture level trade-offs across an entire organization. That's what those big, heavyweight frameworks are for. We were talking about on the ground trade-offs for, huh, I have this microservice architecture, should I do orchestration or choreography as a communication style? I've had conversations with a lot of architects who say, "Oh, you should always use choreography for everything." As an architect, the always word makes my Spidey sense go off that, oh, no always is, oh, that's sketchy.

That's a great example of the kind of trade-off you have to make, but the fundamental problem we have is that- and this is a quote from Mark, you can't Google architecture. When you're a developer, you get really good at Googling answers to questions about configuration and why this is not working and error stacks and which part of the stack to paste into Google to get the best results, but when you're an architect, first of all, what are the chances that the exact combination of forces that you're trying to decide about has ever existed on planet Earth before with the combination of technologies and legacy and all that other stuff?

Even if it has, what are the chances somebody has solved your problem and written a blog post about it that you can find in search on Google? Every trade-off analysis has to be ad hoc now. We actually lay out an incredibly simple framework, which is- and I can tell it to you now out, figure out how interested things are tangled up together. Figure out how changing one of those would affect that tangle and build an analysis from that. Rather than just give you that very simple formula, the entire book is really taking the generic- the two basic major generic problems in distributed architectures, granularity of services and communication, and using this trade-off analysis to do a deep dive into all the different facets of those things.

The wiring of your services, what size should the services be, and how should they call one another? Of course, part of that wiring are a whole bunch of considerations about data and data relationships and schemas, because in modern distributor architectures, of course, you have data as part of a bounded context which makes a lot of those data concerns projected into architecture.

Mark: One of the other things I think as well, Rebecca, that does distinguish this from the older trade-off methods and really introduces a new approach is we really emphasize the fact that in each chapter we build trade-off tables to show the trade-offs of one or the other. However, unlike a traditional scorecard that a lot of people use to say, "Well, this one scores higher than this one so that must be a best practice, or that must be the one I want choose." Rather we introduce those kind of trade-offs, but then put them into a particular context.

Granularity is a great example, Neal, that you just mentioned because for example in that chapter, we go pretty deep on how big should a service be. With there, to give you an example, we talk about disintegration drivers. In other words, all those various drivers and justifications and factors for actually breaking apart a service. Now, people get pretty excited about that part of the book, especially since like our fundamentals book, we've vetted out a lot of this material through conferences and trainings and such to really get good feedback.

People get very excited about those disintegration drivers and say, "Thank you, thank you so much." Code volatility or whether it be scalability or whether it be security, these are great reasons to break apart a service. We actually take it one step further and say, "Yes, but," there's the other side of the coin. There's the granularity integrators that say, well maybe you should keep a service bigger, more coarse grained.

What we really try to strive for in that particular chapter about service granularity is the balance between these two kinds of forces, disintegration drivers and justifications and integration drivers and justifications, and really put it back on the architect, the reader to say, "Thank you. Now I have these opposing forces and I can determine which is more important to me. To isolate code volatility, to make it easier to test and deploy, or transactionality," because we can't have both. Those are examples of how we really changed the game with respect to how to see, identify and actually determine what trade-off should we accept?

Rebecca: I remember there was a very short period of time-- Thoughtworks has this internal software dev mailing list, and at the same time, there was this long thread on, we'd been brought in because they've got 87 microservices and we need to have fewer. There was at the same time another thread, we've been brought in because they have three microservices and they need to have more.

[chuckles] Of course, the specific numbers don't matter, but there are, as you point out, pressures to say, "Actually, no, I need to have a smaller number. I need the granularity to be larger because of this, this or this," or, "Wait a minute, this is far too constrained because of how big these things are, so I do need to disentangle them." Unfortunately in our industry, we keep wanting-- Yes, the perfect number is 12. Obviously, the perfect number does not exist, but it is great that you can talk about, "Okay, here are these different forces, these different drivers. Now, for my organization, for this application at this particular point in time, where should that balance be?"

This is, I think something- the perfect granularity A, doesn't exist, but B, is specific to a point in time. There is nothing that says, even after you very carefully made these trade-offs in nine months' time, you might have to make a different trade-off because these factors have shifted.

Neal: In fact, the common answer to your question is, it depends. What we're trying to do in this book is answer the follow-up question depends on what. Because every time it depends comes up, you can ask the following question, "Well, what does it depend on?" That really is, I think one of the insights that we got from the first book, one of the things in the first book that we tried to do was look at things that seem to apply universally across everything we touched, and we codified those as these two laws. First one is everything in software architecture is a trade-off.

Of course, we doubled down on that in this book, but it really is amazing how much, when you really start pulling on that thread of trade-offs in architecture things, that how much insight you get into how much these things really are trade-offs. Once you've accepted that there are trade-offs, how do you figure out exactly what those trade-offs are? We do a lot of qualitative analysis in this book because it's hard to do quantitative analysis in architecture because you have to have things that you can quantitatively compare, which is really difficult, certainly across architectures, but even within one, but you can do qualitative analysis.

You can say, "If we switched from orchestration to choreography in this case, will this make scalability better or worse?" That is the key to let you do ad hoc trade-off analysis for given situations, for all the forces that are driving you forward in this situation, versus another one, versus another one, because there are no silver bullets in the software architecture world. There is no generic advice that's useful so you have to dig in to- depends on what to get to useful things.

Mark: I think there's also a unique aspect to further answer your question, Rebecca, about how we treat trade-offs in the book and trade-off analysis. Like most things, everything seems to have an anti-pattern or pitfalls associated with it and trade-off analysis is not exempt from that. There are pitfalls and anti-patterns in analyzing trade-offs. As far as I know, ours is the only book that actually addresses those particular anti-patterns and trade-offs. For example, the out-of-context problem is very common when analyzing trade-offs, and that scorecard piece I talked about before. Looking at pros and cons and seeing that there's 20 pros and only 2 cons, so we should do this.

That is a pitfall within analyzing trade-offs and making decisions because it's out of context, as Neal was saying, depends on what related to us. Things like over Evangelism of particular solutions is another trade-off anti-pattern that we get too excited about something and we start- just our brains don't see the negatives because we really like this particular platform or this particular technique or tool. Those are the kind of things that we talk about in the book. There's quite a few of those anti-patterns within trade-off analysis that we cover. I think that also does distinguish how we see and view trade-off analysis today versus yesterday.

Neal: Well, that's one of the things that we very purposely did in the book is the first 14 chapters are written purely in the third person, because we're describing architecture and how to do trade-off analysis. The last chapter, chapter 15 is written in the second person and it's entitled build your own trade-off analysis. We use the 14 chapters before as examples of how to do trade-off analysis, but then give you a bunch of concrete advice as the reader for how to apply the things that we're talking about to your own problems, how to build these qualitative lists, how to build these matrices, and that sort of stuff.

We try to use the examples in the book as a way to get you, the reader, to the point where they can do their own because that's the real value of an architect is being able to do trade-off analysis within your architecture, not for some abstract architecture that exists on Google or on the pages of books somewhere.

Rebecca: We've spent a lot of time talking about one of those two challenges of distributed architectures that you mentioned, which was granularity. What can you say about the communication challenges? Obviously, an architecture diagram that has only boxes is not terribly interesting. You need to know how the boxes interact with each other, but what can you tell me about some of the specific challenges you looked at, with respect to communications?

Neal: We relied on a book that was written back in 1993, called What Every Programmer Should Know About Object-Oriented Design. One of the great insights in that book was the separation of coupling between static and dynamic coupling. Static is the way things are wired together, and dynamic is how things call one another. There are two fundamentally different ways of thinking about architecture, because think about, for example, synchronous versus asynchronous communication.

If you have two independent services to have independent operational architecture characteristics like independent scalability, well, if I make a synchronous call from one of those to the other, I tangle up those characteristics for the course of that call. I don't have independent scalability anymore during that synchronous call. Once that call is over, they're back to independence again, but that entangles them temporarily. Whereas an asynchronous call doesn't have that same effect because you can use asynchronous message cues for buffers and have multiple listeners, et cetera.

There's a fundamental difference there in the way you think about things in architecture, whether it's synchronous versus asynchronous. Communication also has a bunch of decision points within architecture. This is something that we struggled with for a long time. It was how to reconcile all these different forces in architecture. One of the very late insights in this book was thinking about these things as a conjoined three-dimensional space that you can't think about them independently.

The insight here was okay, thinking about transactionality in microservices, you can't just treat that a la carte, because if you change transactionality from eventual consistency to atomic, then it changes the trade-offs in your workflow. Should it be orchestration or choreography because now you've got a different set of trade-offs in your workflow. You have a different set of trade-offs for a lot of things. Flipping those binary switches changes. All these things are entangled, and this is one of our insights.

If you're trying to decide something about communication in these architectures, you have to look at how all the things are entangled and can actually build matrices to say, "If I flip this, what does that impact? Does this have over here and let you do trade-off analysis without having to build every one of them from scratch?"

Mark: Isn't it funny, Neal, do you remember back in the day at conferences you had a very famous analogy that people loved about exactly what you just talking about, and it's actually basically like flying a helicopter. Do you remember that?

Neal: Oh, yes.

Mark: The different controls and axes all impact one another. It's just a perfect fit to this entanglement of all of the aspects of eventual consistency or choreography, and then correspondingly communication. It's almost like the three controls at a helicopter. [laughs] It's funny, Rebecca, when you first asked the question about communication, I had a different mindset, I actually was not thinking about interservice communication protocols. I was thinking about how we communicate our ideas to others. I'm sitting here going "Well, there's a lot of that in our book, but that's more in our first book," but I said- so it's interesting, that association.

However I do think we do touch on that, because it comes back to that trade-off analysis and justification. That requires collaboration and communication with all levels of the organization, all different kinds of stakeholders. Whether it be in the database area, whether it be in operations, whether it be in release management, any of those aspects. That's what I was tying the communication piece and when Neal started talking about sync async I'm like, "Neal, what in the world are you talking about?" Then I realized "Oh, communication protocols." [laughs]

Neal: One of the things that we do in our book is, the saga is the typical name for a transactional workflow within a distributed architecture like this. We have both the figurative saga but we have a literal saga inside our book as well. One of the things that we struggle with because a lot of this material is pretty abstract- and you've written a book about architecture, it's really easy to get really abstract and hard to make it concrete. We realized that this book has a lot of very abstract stuff.

Part of the benefit of teaching this material a lot, we've been teaching this material continuously even through the pandemic, we've been doing online classes and that kind of stuff is that it lets you test out to see how to make things more concrete. During one of our training classes, we came upon this idea of creating this- you're familiar with this idea of architectural katas, these little made up problems that you can solve architectures around. We came up with this increasingly elaborate [laughs] kata around this mythical company called Penultimate Electronics, and the Sysops Squad which is basically, they run around and you hire them to come fix your TV or your computer or whatever.

Throughout the Architecture: The Hard Parts book we have embedded inside every chapter Sysops Squad stories. What that allowed us to do is really have two different levels of abstraction in the book, because we have the exposition abstraction which is the details and the technical, all the details you need an exposition but then it drops into the Sysops Squad and they have a very concrete example they're trying to solve, because they've got a particular problem at the beginning of the book that they're trying to solve. They have concrete problems.

They're applying the things that we've talked about in exposition to solve real problems. This is a problem we faced early on where this book could either be 200 pages, 450 pages or 25,000 pages, because if it was just purely abstract it would be 200 pages and be unreadable, because nobody would be able to read more than two chapters without their heads exploding. At 450 pages, it allows us to selectively pick examples and exemplars to illustrate some of the things without going to every possible branch of the trade-offs. Then the 25,000 page book is let's take every possibility for every choice and follow them to their fullest extent, which again nobody would ever want to read.

This is the classic show don't tell that allowed us to be selective about being abstract but also show concrete implementations for these specific problems as well. I think it worked out really nicely because in my mind struck a good balance between the abstract and the implementation. Hopefully the readers get involved in the Sysops Squad story and hope- wish them well.

Mark: When Neal is talking about story it actually is a story, complete with dialogue and characters. One of the I guess, advantages that both Neal and I have of- well, being quite a few many years in the industry mine going on 36. I think Neal is little more than that, but close, is the fact that we through our consulting experience have seen a lot of scenarios, and it was amazing as we're writing this dialogue, I was essentially pulling actual meetings and such and actual experiences that I had all the way past in my consulting career. It made writing that dialogue a lot easier, and also it made it more real.

Rebecca: All the names are made up, to protect the innocent.

[laughter]

Neal: I expected many of those stories were cathartic for Mark to write all those things down. I expected many of those stories were cathartic for Mark to write all those things down. Speaking of names, one of the very intentional things that we did in this book, because we realized that there's a long history of accidental misogyny in technical books, so every single character in this book has a gender-neutral name, very much on purpose. They also have alliterative names for their role. All the architects have A names. We have Austin, Addison, Logan is the lead architect. That's an L name, and there's not a single gendered pronoun in the entire book.

You will appreciate this approach, Rebecca. I actually built a fitness function that would scan the entire manuscript, looking for gendered pronouns to make sure that we didn't accidentally add one at some point during the writing process. We were very careful about that, because it's time. One of the things that we wanted to do is make sure that you could write a book like that and not use any gendered pronouns and not make it sound and read in a stilted way. I don't think it does at all. I don't think you notice the absence of this thing. That was actually a little bit of a writing experiment that turned out really super well.

There are a couple of other little Easter eggs, in the Sysops Squad story stuff. One of them is the gendered pronouns and the names, but another is we wanted to come up with a way to separate each of the Sysops Squad stories from the main text. We came up with this idea of using timestamps. Every Sysops Squad story starts with a timestamp with a date and a time, but not a year. There's a slightly jumbled chronology of those timestamps. If you pay close attention to them, the very first timestamp of the book happens on September 21st. The last one happens on June 20th.

Of course, that is the equinox. The book begins on the first day of the year where night is longer than day, and it ends on the day of the year where you have the longest daylight. There's a little bit of a time-based thing going on there too. There's one other little Easter egg there that nobody will pick up on. The very first timestamp in the book is for 1,300 hours. If you remember the book 1984 very strikingly starts out with saying, "It was a cold day in April and the clock struck 13," which indicated something was very wrong in that world. The very first timestamp is 1,300 hours in the-- [laughs] There was a lot of little things like that scattered throughout the book.

Rebecca: Neal has asserted in the past that it takes two years to write a book. [laughs]

Neal: It always does. Well now you see why, because you can't stop fiddling with stuff like that.

Rebecca: One might also comment on the fact that perhaps you're a little bit of a science fiction geek, but far be it for me to make that sound like that's a bad thing because I'm a self-proclaimed geek. [chuckles]

Neal: Well I think I captured my core demographic well with that pretty easily. [chuckles]

Rebecca: One of the things that I particularly enjoyed as we were working on the evolutionary architecture book were some of the things that just- we discovered, the architectural quantum, that whole notion came after we had been talking about this for a long time. Can you tell about some of the discoveries you all made in this particular book?

Mark: One of the ones that stand out most to me from these innovative ideas, was what we came up with from the concept really of transactional sagas. There is a core transactional saga pattern. What Neal quickly realized was there's dimensions, as a matter of fact, three of them that really form a transactional saga which mathematically points to eight possible combinations. I think, wouldn't you agree, Neal, that was probably one of the more innovative aspects within the book, one of many.

Neal: I remember like a lightning bolt when it finally came to me, I was listening to Mark talk about workflows for probably the 50th or 60th time in the training class that we do together. It finally hit in my head that you can't change this without changing the other stuff and realize that these are conjoined in a three-dimensional space. We have an actual three-dimensional diagram there that if you move the ball here, it affects the other parts of the space. That was a really interesting insight.

What was interesting about that to me was I had gotten really close to that insight almost a year before, and then abandoned it as a dead-end and went off in another direction and then eventually came back around to that. It had been percolating in my head for a really long time. Finally listening to Mark talk about it yet one more time was the trigger that finally made it congeal in my head.

Mark: One of the other innovations there which finally we were able to actually document in a book were various component-based decomposition patterns, which I started to develop back almost about six years ago in decomposing monolithic applications to microservices. That chapter is up I think originally, Neal, wasn't it about 65 pages long? Was, and I think still is, the largest chapter in the book. We considered breaking it up, but it's one cohesive thing, but these are approaches, very incremental controlled approaches to basically breaking apart a monolithic application.

I was so happy to finally get that documented somewhere after the years and years of refining those patterns, but also showing these patterns at conferences and really getting feedback and being able also through regular consulting, to actually exercise these patterns and fine-tune them. I think that's another major innovation or innovative part of the book that really addresses that breaking things apart in part one. As a matter of fact, a piece of trivia here, that Neal and I actually-- Actually, Neal saw this during the review was the book really is separated into two parts, unlike our first book, which was three.

It's really about part one, which is decomposition, breaking things apart. Part two is really about putting them back together, interconnection and communication. What Neal observed, which was really cool, drumroll, and I'll let Neal tell this really cool aspect, little trivia.

Neal: The final book manuscript ended up being 450 pages, and part two began on page 223. We almost exactly split it down the middle, and absolutely no effort whatsoever to do that. In fact, that first delineation between pulling things apart, versus putting things back together was purely an exercise in the first time, the very first time I did a talk called Software Architecture: The Hard Parts. All these examples that I- I need to put these together in some sort of cohesive talk, how are these things related to one another?

There were two buckets of, well these are kind of structural and pulling things apart, and these are kind of communications. I came up with that. It actually started as a three-part thing, but then it quickly collapsed into the two parts that are in the book, but it's just amazing and shocking to those of us who wrote the thing that it came out almost exactly half and half.

Rebecca: What was the third part that was there for a little while, the third bucket?

Neal: The third, it was pulling things apart, putting them back together with appropriate patterns, I think was the third part. It was simple parts. That's right. It was simple parts. Exactly. There's ways to put things together without overcomplicating them, but that quickly becomes just putting things back together properly. [laughs]

Mark: Approximate projected page count of that was almost 225 pages. This would have been a book that would be too heavy to carry around. Like Neal said, we would have been just gone all out and gone to 25,000 pages.

[laughter]

Neal: Well, in this book, we expected was going to be shorter than the fundamentals book, but it ended up being 20 pages longer because there's a lot of stuff it turns out as you start pulling on that thread of structuring communication, particularly as you start overlaying the data stuff. That was the other great insight that Mark and I had as we finished up the fundamentals book is that, look, modern architectures you can't treat data architecture and software architecture as separate things anymore. It was a mistake to do it before, but you really can't do it now.

That's why we included Pramod and Zhamak as co-authors of this book, to talk about the twin ideas of operation in analytical data. We won't talk a lot about that subject matter here because there's going to be a separate podcast with him just talking about the data but that was the other insight that we had is you can't really do a book about distributed architectures now without considering the impact that data, both structurally and communication has.

Rebecca: Absolutely. Data used to be that thing off to the side and you just can't think about it that way anymore. I almost hate to ask this question, but what's the next book?

[laughter]

Neal: Well, that we don't know yet, because one of the main ways that we generate new ideas like this is speaking at a lot of conferences, and of course we haven't been able to do that a lot. The Hard Parts came as a very natural extension because we had this massive pile of things, and we didn't exhaust the pile by the way. Mark just out of the camera range that we can see on Zoom now has a giant pile of topics right behind him on distributed architectures and implementation patterns for distributed architectures and things like that, but we've also got a ton of material on enterprise architecture and some topics like that.

That's going to be one of the next tasks that we do, is get together and figure out exactly what the next thing is going to be. We have thought about doing software architecture, the harder parts, and then of course, that would lead to eventually software architecture, the hardest parts, would be an entire book about the non-technical parts of software architecture, soft skills, organization, meetings, because that it turns out is the hardest part of software architecture, but we'd have to write the harder part to get to the logical hardest part. We don't know what exactly that's going to be.

Rebecca: Yes, let's see, two years, that might be a while.

Neal: Exactly. [laughs] We did actually manage to get this one done in slightly less than two years, though. We accelerated a little bit because it was February 2020 to November 2021. Almost two years, just under, but we'd started working on it before the fundamentals book came out, so we cheated a little.

Rebecca: Well and both of your travel schedules were a wee bit disrupted during that time, that might have something to do with being able to do it a bit more quickly. A pleasure as always, Neal and Mark, to talk about software architecture. Thank you so much for spending the time with us again, and when will the book be available?

Neal: By the time this podcast is out, it went to press literally yesterday as we record this, and so the ebook will be available by early October, and the physical book will be available early November.

Rebecca: Excellent.

Neal: Available at all fine booksellers near you. As I tell people, it's an O'Reilly book so you should get two in case you lose one. They make great Christmas gifts and birthday gifts and anniversary gifts. Nothing says loving your spouse like a nice O'Reilly book.

Rebecca: You were born a marketeer, Neal. Thank you all for joining us on the Thoughtworks Technology Podcast.