Training a large language model like ChatGPT-3 consumes gigawatts of energy on a gigantic computer, resulting in about 500 tons of greenhouse gas pollution.
Experts say even using generative artificial intelligence to respond to user prompts would require orders of magnitude more power than Google Search.
As such, machine learning can be used for tasks such as climate modeling and integrating a higher share of renewable electricity into the grid, but researchers say the use of the technology comes with climate change costs. increase.
This issue raises questions about the appropriate use of this technology and the potential for smaller, more efficient models that can deliver the same performance with less power.
Professor Simon Lucy, Director of the Australian Institute of Machine Learning, University of Adelaide, describes the environmental impact of machine learning as a ‘two-sided coin’.
“There are many public benefits.
Lucy said the energy consumption issues of machine learning are similar to concerns often raised with cryptocurrencies.
“Basically, these giant computers are consuming gigawatts of energy, and depending on where the energy source is, they can create a lot of carbon in the atmosphere,” he explains.
As with large language models, for generative AI, the training phase is the most energy-intensive as the system searches for data. But the system still consumes a lot of energy when operated by the user, says Lucy.
Training and operating machine learning models is energy intensive and carbon footprint intensive
According to Stanford University’s AI Index Report 2023, greenhouse gas pollution emitted by machine learning models depends on factors such as the number of model parameters (data points), the energy efficiency of data centers, and the power source to those centers. It is said that
In general, the larger and more complex the model, and the dirtier the electricity used to power computers and data centers, the higher the resulting greenhouse gas emissions.
In the report, Huggingface research scientist and climate change leader Sasha Luccioni analyzes emissions from training four large-scale language models (Gopher, BLOOM, GPT-3, and OPT). is quoted.
GPT-3 produced the most emissions (502 tons CO)2), 20 times the lowest emissions BLOOM (25 tonnes of CO2)2). But BLOOM’s training also used the equivalent of powering an average American home for 41 years.
Newer models like GPT-4 are even larger in terms of model size and number of parameters.
| model | parameter | Data Center PUE | grid carbon intensity | power consumption | CO2 equivalent emissions (EE) | CO2EE×PUE |
|---|---|---|---|---|---|---|
| gopher | 280B | 1.08 | 330g CO2ee/kWh | 1,066 MWh | 352 tons | 380 tons |
| bloom | 176B | 1.20 | 57g CO2ee/kWh | 433MWh | 25 tons | 30 tons |
| GPT-3 | 175B | 1.10 | 429g CO2ee/kWh | 1,287 MWh | 502 tons | 552 tons |
| opt | 175B | 1.09 | 231g CO2ee/kWh | 324 MWh | 70 tons | 76.3 tons |
Is using such an energy-intensive algorithmic system appropriate for the task?
Friederike Rohde, who studies the social and environmental impacts of digitization and machine learning, recently co-authored a paper on the sustainability challenges of artificial intelligence, Ökologisches Wirtschaften (scientific journal of social and ecological economics)).
We should be careful not to get trapped in unsustainable infrastructure, Rohde said, and given the high energy and emission costs of machine learning, we should educate people about the adequacy of using AI across different fields and tasks. He asks you to think carefully.
“Do we really need every sector and everything to put machine learning behind every tool?” she asks.
“Is using such an energy-intensive algorithmic system appropriate for this task?”
These questions are timely as technology companies such as Meta, Microsoft and Alphabet (which owns Google) have already said they are integrating generative artificial intelligence tools into their workplace tools and social media platforms.
“AI has reached a point where we have to really think about how it is used and how it is used responsibly,” says Lucy.
Responsible AI, he said, is about ensuring that the technology is applied where it can do more good than bad.
But he predicts that concerns about energy use and emissions won’t last long. Tech companies and research institutes are increasingly thinking about where they get their power from, he said.
And ultimately, energy costs will drive technology development to make software, and hardware, more powerful and efficient.
“One of the reasons many companies in the machine learning space are losing money today is energy consumption, because the cost of actually running the queries is much higher than the revenue they generate from servicing the queries.” he says.
One way forward is smaller models with fewer parameters, an approach that open source developers are already working on. “You get it really small and really fast,” he says.
“I can now run the equivalent of GPT-3 on my laptop, which would have been unthinkable even six months ago.”
Can machine learning be used to solve environmental problems such as climate change?
Lucy thinks so.
For example, machine learning can help derail the power grid by better predicting weather conditions, predicting and integrating fluctuating renewable energy sources, and harnessing technology’s ability to make energy use time more efficient. It can be used to support carbonization, he said.
Another example is the NOBURN app developed by the Australian Institute of Machine Learning in partnership with the University of the Sunshine Coast. This citizen science tool collects data from people living in fire-prone areas and uses machine learning with the ultimate goal of better predicting the likelihood of wildfires and mitigating their impact. and
Lucey believes that machine learning will improve productivity rather than replace the critical role of humans in making decisions in high-risk scenarios.
Rohde agrees that there are applications of machine learning that can have a positive impact on sustainability, and that increasing the deployment of renewable energy can be an effective use. Agree.
But she warns, “I think we should be cognizant of this narrative that machine learning tools can help us take a really big step towards sustainability.”
“One of the reasons many companies in the machine learning space are losing money today is energy consumption, because the cost of actually running the queries is much higher than the revenue they generate from servicing the queries. is.”
Professor Simon Lucy
In other areas, the technology has not proven useful, she says.
The key issue is that machine learning trained on historical datasets fails to produce innovative solutions and can bias the status quo.
For example, AI algorithms may help optimize city traffic flows, but they are not designed to produce innovative solutions, such as changing entire transportation systems to reduce reliance on cars. yeah.
Rohde added that many climate and environmental issues also depend on social processes, from the decisions that society takes seriously to the implementation of measures.
“Applying machine learning can tell us what the problem is, but solving it requires a social process,” she says.
“We have to make political decisions or companies have to take responsibility for what they are doing.”
