Developments in generative AI have come at a particularly rapid pace over the past twelve months. It isn’t surprising, then, that alongside optimism and enthusiasm there is also concern about risks, ranging from deep fakes, to lost jobs, to threatening human civilization. The environmental impact of this technology has also attracted attention. However, not enough is currently known about AI’s carbon footprint – and that means it can be hard to know exactly what we should be doing about it. What should we be worried about, exactly? Are we worrying too much?
One thing’s for sure: if we’re to make AI safe and sustainable, we need to think carefully about how we monitor, communicate, and mitigate the technology’s environmental impacts.
What’s the true environmental impact of generative AI?
First and foremost, it’s essential that we gain a clear picture of what the true environmental impact of generative AI actually is. This is, of course, complicated — and, given the amount of interest and commentary on this emerging field, making sense of the information available is very difficult.
A good place to begin, though, is to recognize that AI’s environmental footprint comes from different sources — in the case of large language models (LLMs) like ChatGPT, this includes first the training of the enormous models which form the basis of many products and solutions, and secondly, the actual serving and use of those products has its own environmental impact. As users or deployers we may have less control over the first element than the second, yet we nevertheless need to acknowledge that we make an active choice, first about whether to use generative AI at all, and then which specific model or product we choose.
It’s also worth putting things into perspective. Although the technology industry accounts for between 2-4% of global greenhouse gas emissions, AI is only a fraction of that total. Training large AI models, for instance, is something that only happens with relatively low frequency — maybe there have been 1000 LLMs trained so far. Emissions from transport and manufacturing are constant, happening all the time. And while it’s likely that emissions from the use of AI products and services will increase as adoption grows, at present the scale is vastly different. In 2019, for example, aviation produced 920,000,000 metric tons of CO2, while one analysis suggested ChatGPT could emit 15,000 metric tons over a twelve month period.
The importance of transparency
Measuring and managing the environmental impact of AI requires transparency. Meaningful transparency allows practitioners and organizations to make informed decisions about when and how they use AI — at the moment, it is exceptionally hard to connect implementation to environmental impact.
Sara Bergman, a Senior Software Engineer at Microsoft, has urged people to consider the entire lifecycle of an AI system, rather than just one particular aspect of it. It requires us to think through both the impact of the training of the model we’re using, how we’re then using it, how users or customers might be interacting with our associated product or service and even easy-to-miss yet no less significant consequences of our wider technical architecture and ecosystem.
Thinking in this way now is important: it’s not unlikely that we’ll begin to see legislation in this area. In France, a law was recently passed which mandates telecoms companies to be transparent when reporting their sustainability efforts — seeing this applied to AI systems is certainly not beyond the realm of possibility in the next few years.
How can we begin reducing GenAI’s environmental impact?
Fortunately, there are many ways we can reduce the environmental impact of GenAI. Although there’s no single silver bullet, using multiple techniques and approaches can go a long way in ensuring GenAI is carbon efficient and more environmentally friendly than it otherwise would be.
Indeed, improvements in the performance of LLMs is, at present, largely built on increasing the size of the models on which the systems are trained. While scale may improve the capabilities of LLMs, they inevitably require more computational power to actually run. The techniques and approaches are all valuable ways of mitigating the environmental impact of generative AI.
Quantization and distillation
Quantization is an approach that reduces computational intensity by sacrificing some precision and accuracy in models. Doing so can mean that running models doesn’t require powerful, specialized hardware — quantized models are sometimes used, for instance, on consumer-level hardware, like a laptop. While it might seem strange to sacrifice accuracy — surely that’s a step backwards, right? — if a particular use case doesn’t require such intense levels of precision it can make a lot of sense. Indeed, not only will it reduce the environmental footprint, it can also reduce the costs involved.
There are other related techniques, such as distillation, where a larger model is used to train a smaller one. Although the training of two models could increase carbon emissions, this is compensated by reducing in-use emissions.
Optimizing model architecture
Just as the logic of quantization is essentially about sacrificing some level of precision to reduce intensity, we can also take a similar approach to models themselves — in other words, choosing to use a simpler model over a complex one. Complexity is typically the result of trying to get generative AI models to do more, to become more generalizable: that’s partly the attraction of ChatGPT and the excitement about what might be possible with this technology (ie. an AI that can do anything or answer any question). However, if your use case and context can be more tightly defined, you could develop a model that is much less complex than state of the art models but just as effective.
Green prompt engineering
The concept of prompt engineering has quickly followed GenAI’s ascent — some have even gone as far as to call it a job of the future. Whether that’s true or not remains to be seen, but it is worth noting that the way a generative AI system is prompted (ie. questions or requests are phrased to the system) has an environmental impact. This might, at first glance, seem surprising, but the way to understand it is that the longer a prompt, the more information (processed as a token by the system), the more computational power the system requires. So, shorter prompts should be more environmentally friendly.
Of course, shorter prompts may make it harder to get the output you want — good intentions could easily lead to worse consequences if you end up needing to provide more prompts. However, this is why prompting needs to be approached scientifically: getting the balance right so you can achieve the intended goal in as efficient a manner as possible. Application of an appropriate evaluation framework can allow efficient experimentation with prompting, providing clear metrics for performance, cost, and carbon emissions.
Other techniques to consider
Although the considerations discussed above have largely centered on software design, there are also more material things that we need to pay attention to. The hardware you select, for instance, will have a carbon footprint and it’s also possible to use ‘cleaner’ and more environmentally-friendly electricity (this is because electricity comes from different sources at different times of the day according to demand). These considerations are both discussed in depth by the Green Software Foundation, an organization that Thoughtworks is proud to be a founding member of.
Concluding remarks
There’s no one single solution to the carbon emissions of artificial intelligence. However, at a time when climate disaster is the key issue facing the planet, it certainly wouldn’t be responsible to embrace the possibilities and opportunities of AI without thinking through the environmental impacts of the technology and taking steps to mitigate them. We believe it's imperative that emission impacts are considered first-class metrics in the design and development of AI systems. All steps add up to have a larger impact, and some immediate steps include learning about and applying the principles of green software, downloading the open-source tool Cloud Carbon Footprint to start analyzing your cloud footprint, and joining the Green Software Foundation to influence policy and standards to measurably mitigate the environmental impacts of tech.
Further, use of a modern CD4ML or other MLOps system can allow integration of carbon metrics directly into the development process. Although these approaches are not yet fully developed in existing tooling, Data Scientists and engineers can and should have immediate feedback as to the carbon implications of their design decisions, and models’ carbon emissions should be tracked and monitored. Making this easier is a critical next step towards the ideal of green AI.
While the approaches and techniques mentioned above are neither exhaustive nor likely to solve AI’s environmental challenges on their own, they do represent a valuable starting point as the industry continues to experiment with GenAI.
