Macro trends in the tech industry | May 2018

Twice a year we create the Thoughtworks Technology Radar, an opinionated look at what’s happening in the enterprise tech world. We cover tools, techniques, languages, and platforms and we generally call out over one hundred individual ‘blips’. Along with this detail, we write about a handful of overarching ‘themes’ that can help a reader see the forest for the trees. In this piece, I try to capture not just Radar themes, but wider trends across the tech industry today.

The 'half-life' of change shrinks once more

It’s become a frequent refrain in the tech industry — the pace of change is accelerating, you need to react faster, and things that are with us today will be rapidly eclipsed tomorrow. We see this reflected in the Radar in a very concrete way: we force fade blips on the Radar far more often to make room for new stuff, with an increasing number of blips only surviving a single volume (that’s just six months of life for a blip). I’ve long argued that a 21st-century organization must rapidly adapt and absorb new tech in order to survive — the reduced half-life of change on today’s Radar seems to confirm that.

Evergreen architectures

Continuous Delivery has been a key enabler to get software into production faster, but we’re also seeing it bear fruit in enabling Evolutionary Architectures. With a cloud platform and good automation techniques, we can actually treat software architecture as code — infrastructure, networking, servers, services and interconnections all defined and managed through version controlled text files. This enables us to make architectural leaps that were previously difficult or impossible. As with the Ship of Theseus, we can continuously replace parts of our system until the whole thing is new, and then keep going as necessary or strategic. We helped one of our clients, a large European e-tailer, introduce Continuous Delivery and then move from JAR files hosted in Tomcat, to Mesos, and finally to AWS. Major changes, but it hasn’t hurt them during the migration.

Microservices in Adopt?

Despite having this architectural style around for years (we wrote the definitive article, a book, and a guide to breaking a monolith into microservices) we’ve never actually moved microservices into the Adopt ring of the Radar. Our CTO Rebecca Parsons has written a detailed piece on this, but I’ll cover the high level here. Firstly, remember that Adopt doesn’t mean “always use this and never the alternatives” — just because we put Clojure in Adopt it doesn’t mean “abandon all other programming languages” (unfortunately!). Our placement of microservices in Trial is quite deliberate: we think it’s bad advice to say “always use microservices.” It’s very hard to migrate from a monolith to microservices; it’s hard to decide when to split services, it requires a good underlying platform and great DevOps practices. It’s a good potential destination but it’s a long journey. We often see people end up with a distributed monolith — they distribute their system but end up with none of the benefits, mostly because they don’t get the domain modeling right and fail to really adopt the microservices architectural style: they just have small services.
 

Machine Learning slowdown?

We’re not talking that much about Machine Learning in this edition of the Radar. The absence of it as a theme is, in fact, a theme — ML is moving through the Gartner hype cycle, down from the “peak of inflated expectations” through the “trough of disillusionment” to the “plateau of productivity.” And this is in fact what we observe in the market and at our clients: the time of one breakthrough after the other going through the press and social media is over. The vendors are working to complete their offerings (see the blip for TensorFlow Lite) or make them rock-solid for production usage. Organizations are starting to consolidate all the different proofs of concept which popped over the last couple of years, with tools, platforms, and services on one side and inflated expectations perpetrated by media and AI and software vendors on the other side. Now Business and IT are discussing the questions: Where can AI bring which benefit to the organization? What software, tools, and platforms are the right ones for the organization? What skills do I need in-house or to buy from the market? These are the right questions to move in the direction of the “plateau of productivity.”

Browser bulks up; server slims down

Despite the rise in mobile platforms such as iOS and Android, web-based UIs still dominate the industry, with a high demand for richly interactive user interfaces. Simple “HTML+forms” based UIs have been replaced by complex Single Page Applications, with large amounts of state and logic running in the browser. At the same time, server-side logic has been reduced, often to a series of APIs that serve the needs of the browser. This trend continues with several new technologies that allow the browser to do even more. WebAssembly is a binary format that opens up new language options such as C++ and Rust running directly (and safely) in the browser. Web Bluetooth opens up functionality previously reserved for native applications. We expect this trend to continue: display and interaction logic resides in the client, platforms or sidecars handle cross-functional concerns, and back-end services can focus purely on core domain logic.

Cloud gets complicated

This edition of the Radar includes a significant number of AWS-related blips, and we culled a lot more during our discussions. With AWS featuring so heavily, is this an indication of its maturity, a genuine market preference for AWS, or the result of Amazon’s explicit strategy to out-compete other cloud platforms based on features? We think it’s a combination, but many retail organizations are eschewing AWS because they view Amazon as a competitor and have made a strategic decision not to use their services. We see a growing confidence in Azure and GCP, both of which are maturing as platforms and whose messaging is resonating with buyers. There is a lot of choice in cloud, but with new data protection laws such as the EU’s GDPR coming into effect, it’s becoming harder for organizations to identify which cloud will support their regulatory and compliance obligations. Today, more than ever, cloud is much more complicated than an edict from the CTO to “put everything in the cloud.”

Kubernetes is an “app server for the cloud era.”

While discussing the themes for this Radar, someone posed the question: “Will we, in three to five years, be angry at Kubernetes in the way we’re currently angry at WebSphere?” Will it become the ‘app server behemoth’ of the future, with everything coupled to it, preventing us doing incremental technology refreshes? Will Kubernetes be the new middleware, tangling our systems into an intricate, Google-powered ball of mud?

The consensus around the room is that we’ve actually dodged a bullet here — many of the PaaSes were looking like being “WebSphere 2.0”, but Kubernetes pulled back from this complexity with a simpler platform and model.


Back in 2005, Bruce Tate wrote an entertaining blog post about his new pet monkey Joe (after a coffee-stain on his chest) that is much better than his old pet Sea Bus Bus. The cute little monkey is a great pet, to begin with, but ends up eating a book about XML and becomes a Gorilla, pushing him around and wanting to be involved in every conceivable project. Bruce’s point is that technology expansion is inevitable. So should we expect Kubernetes to become bloatware in future? Maybe not. Ever-increasing complexity is driven by vendors satisfying their need to build more and bigger products. It’s different with Kubernetes (originally from Google, now managed by the Linux Foundation) because it’s not from an IT tools product company. Tools, frameworks, and platforms are increasingly coming from practitioners or companies that (for whatever reason) make their own stuff available. Kubernetes is fundamentally different from appservers in this regard.

But if we think about the decreasing half-life of change, will it even matter in five years? We can now radically change our architectures in an afternoon, just by changing the configuration of how we deploy our software and components. With software-defined everything, we can maneuver at a faster speed. Things like Continuous Delivery have changed the landscape — there will always be a better new thing, but now it’s easier to adopt it. This kind of rapid, continual evolution is what keeps me excited about working in software.