Large language models (LLMs) are widely viewed as having the potential to transform the productivity of companies due to their exceptional language processing capabilities. However, although the potential of LLMs is exciting, there are notable limitations to their current capabilities.

This means they need to be used thoughtfully, with a clear, systemic and adaptive strategy. Moving quickly to embrace LLMs might be tempting, but you need to do so in a way is safe, secure and complements your team.

But what does that involve exactly? In this blog post I’ll explain how you can leverage LLMs to gain a competitive advantage while avoiding common pitfalls.

Understanding what LLMs are — and what they’re not 

If you want to successfully leverage LLMs in a business context, a good place to begin is to gain a robust understanding of what LLMs actually are. Given the hype around LLMs, and the often wild claims about them (such as sentience), it’s important to remain focused on what is and isn’t possible with LLMs. I recommend this beginner friendly dive into how LLMs work.

Although the term LLM might seem incredibly new, they’ve actually been used in the industry since 2018, when OpenAI released their first generative GPT model enabled by the findings of Google’s “Attention Is All You Need” paper from 2017. It’s also worth noting that an LLM isn’t an application or platform in the way that ChatGPT is — it’s a machine learning model that takes text as an input and produces statistically likely text as an output.

The graphic below demonstrates how an LLM fits into a technology stack.

Technology layer of an LLM-powered application

Think of LLMs as a component of software for natural language understanding and generation. By integrating LLMs, you can add advanced language processing capabilities to your applications, many of which would have previously been impossible.

When used effectively LLMs can enhance applications, and create value for both customers and internal stakeholders. They can, for example, enable a natural language experience for users that are faced with vast amounts of unstructured text data.

LLMs are not a shortcut to business transformation

Given that every business deals with language in their day-to-day operations, opportunities for LLMs are real. However, it’s important to remember that processes often involve multiple consecutive steps where language is not processed or LLMs cannot be utilized due to their limitations. This makes achieving a comprehensive, company-wide AI transformation challenging.

Let’s take a look at some of the LLM limitations.

• Limited capabilities: LLMs are great for tasks that actually require text generation, like copywriting, but fall short in predictive tasks, like text classification, compared to non-generative language models trained for the task. Some may also try to use LLMs to automate tasks that require human-like reasoning and planning capabilities, but even though LLMs are trained to produce plausible-looking reasoning, they still lack true reasoning and planning ability.

   

• Reliability of outputs: For many use cases, one important factor is how reliably the LLM component correctly generates what was requested when embedded into your applications and processes. Although various prompting techniques, retrieval augmented generation, and post-processing guardrails can be used to reduce confabulations, achieving sufficient reliability in mission-critical use cases remains a real challenge. Also, LLMs are usually not deterministic components, which means they may produce different outputs for the same input at different times.

   

• Generation latency: This can be anything from a couple seconds to tens of seconds with current public LLM products like OpenAI’s GPT APIs. This becomes an even more significant factor when multiple LLM and added external plug-in requests are needed. In chat-based UIs, making the user wait for a few seconds might be acceptable but LLM opportunities extend beyond just chatbots.

It’s worth considering the case of autonomous LLM agents, a software system which autonomously tries to complete tasks needed to achieve a given goal, as an example of an ostensibly exciting but currently limited innovation. Although they’ve caught significant public attention, when you combine all the limitations mentioned above, the reality is that even today's best GPT-4 agent performs only 15% of tasks successfully compared to humans, who have a success rate of 92% in a benchmark designed for general AI assistants. Embra, an early LLM agent startup, also recently announced that they are pivoting away from autonomous agents due to the unreliability and slowness of LLMs in automation use.

Finding business opportunities with LLMs

While LLMs won’t immediately revolutionize entire businesses, they can nevertheless serve as valuable tools for streamlining language-related tasks. For example, we used an LLM solution to help Bolt.Works augment their recruiters’ workflow by transforming vast amounts of unstructured text data into structured format. Elsewhere, Nestle and Unilever are augmenting their marketing with LLMs, while here at Thoughtworks, we developed an LLM-powered strategy ideation and concept generation tool.

These real-life examples demonstrate that companies are seeing success when applying LLMs to their processes across various functions. As an organization’s AI and LLM maturity grows with every new implementation, there’s potential to reinvent larger parts of the business, rather than just revamping existing steps. Becoming an AI-first company can lead to new exciting business opportunities but to do so, you need to adopt a systemic and adaptive approach to AI strategy and innovation, as Andy Nolan wrote in this blog post.

All in all, the current generation of LLMs are better at augmenting human capabilities at work rather than widespread automation of tasks. Automation is all about making things efficient, fast, reliable and with minimal random variation, such as on factory assembly lines. In turn, augmentation is all about supporting humans to be more creative, productive and to learn new things — like having an enlightening sparring session with a colleague. This is where current LLMs can be the most effective.

Experiment with available services, gain advantage with custom models

When the gold rush of integrating LLMs and crafting special prompts is running hot, it’s good to remember that everyone has access to the same public LLM services, like OpenAI and Anthropic, and clever prompts won’t be a long-lasting advantage.

The only way to gain a real competitive advantage with LLMs is with your proprietary business data. For example, you can augment the LLM with your own data to generate more factual and tailored outputs for your business. Another option is to use your data to fine-tune smaller models tailored to the specific needs of your business, which requires higher maturity in data and machine learning capabilities but provides advantages like cost savings and accuracy gains in the long run. You own and guard your data assets, so true ownership of your AI models is equally important too.

While publicly available LLM services won’t give you a competitive advantage in and of themselves, they still have their place. These services offer easy, quick and relatively cheap options to experiment with different natural language understanding and generation tasks before diving into training custom models that build on what is distinctive for your organization. If you lack the required data for training custom models you could even generate training data with publicly available LLMs.

Conclusion

By being pragmatic, businesses can unlock the true potential of LLMs and use them to create value for both their customers and internal operations. Just remember to focus on:

• Augmenting your people’s capabilities at work

• Utilizing proprietary data with LLMs

• Experimenting with existing LLM services

• Building customized models for a sustained competitive advantage

Embracing LLMs thoughtfully and with a clear, systemic and adaptive strategy can position your business at the forefront of the competition, regardless of the current hype cycle.