Power grid creaks as demand for AI soars

Sasha Luccioni of machine learning company Hugging Face says there's a big problem with generative AI. Generative AI is energy-intensive.

“Every time you query a model, the entire model is activated, which is very inefficient from a computational standpoint,” she says.

Consider large-scale language models (LLMs), which are at the heart of many generative AI systems. They are trained by accumulating vast amounts of written information and can churn out text in response to virtually any query.

“With Generative AI… you’re generating content from scratch, essentially creating an answer,” explains Dr. Luccioni. That means your computer has to work pretty hard.

A recent study by Dr Luccioni and his colleagues found that generative AI systems can use around 33 times more energy than machines running task-specific software. The study has been peer-reviewed but has not yet been published in a journal.

However, it's not your computer or smartphone that consumes this much energy. The computations we increasingly rely on take place in vast data centers that most people don't see or even notice.

“The cloud,” Dr. Luccioni says. “You don't think about these huge metal boxes that get hot and consume so much energy.”

Data centers around the world are using more electricity than ever before. It will consume 460 terawatt-hours of electricity in 2022, and the International Energy Agency (IEA) predicts that this will double in just four years. Data centers could use a total of 1,000 terawatt-hours per year by 2026. “This demand is roughly equivalent to Japan's electricity consumption,” he said of the IEA. Japan's population is 125 million.

Data centers store vast amounts of information, from emails to Hollywood movies, for retrieval from anywhere in the world. Computers in faceless buildings also power AI and cryptocurrency. They support life as we know it.

The head of National Grid said in a speech in March that electricity demand in UK data centers will increase sixfold in just 10 years, largely due to the rise of AI. However, National Grid expects the total energy required for transport and heat electrification to be even greater.

U.S. utilities are starting to feel the pressure, said Chris Saple of consultancy Wood Mackenzie.

“They're being hit with data center demand at the same time that domestic manufacturing is experiencing a renaissance, thanks to government policies,” he explains.According to US reports, lawmakers in some states are now reconsidering tax breaks offered to data center developers, as data center developments are putting a strain on local energy infrastructure.

Seiple said “land grabbing” is occurring in locating data centers near power plants and renewable energy hubs, adding, “Iowa is a hotbed for data center development and wind power is thriving. ” he said.

These days, some data centers can afford to move to more remote locations because latency (the delay, usually measured in milliseconds, between sending information from the data center and when you receive it) is not a big concern for the increasingly popular generative AI systems. Historically, data centers that handle, for example, emergency communications or financial trading algorithms have been located in or very close to large population centers to achieve the absolute best response times.

Data center energy demand will undoubtedly increase over the next few years, but Seiple stressed there is a lot of uncertainty about how much it will increase.

Part of that uncertainty lies in the fact that the hardware behind generative AI is constantly evolving.

Tony Grayson, general manager of data center business Compass Quantum, points to Nvidia's recently released Grace Blackwell supercomputer chip (named after the computer scientist and mathematician) as an example. The chip is specifically designed to power high-end processes such as generative AI, quantum computing, and computer-aided drug design.

Nvidia says that in the future, it will be possible to train an AI several times the size of the largest AI systems available today in 90 days using 8,000 previous generation Nvidia chips. This will require a 15 megawatt power supply.

But Nvidia says it can do the same job with just 2,000 Grace Blackwell chips simultaneously, requiring 4 megawatts of power.

Still, the final power consumption would be 8.6 gigawatt hours. This is about the same amount that the entire city of Belfast uses in his week.

“We have significantly improved performance, resulting in significantly higher overall energy savings,” Grayson says. But he agreed that power demand is changing where data center operators set up facilities, saying, “People are going to go where the power is cheaper.”

Dr. Luccioni points out that manufacturing modern computer chips requires a great deal of energy and resources.

Still, it's true that data centers are becoming more energy efficient over time, argues Dale Sartor, a consultant and affiliate of Lawrence Berkeley National Laboratory in the US. Its efficiency is often measured in terms of power usage effectiveness (PUE). The lower the number, the better. He notes that his PUE for a state-of-the-art data center is about 1.1.

These facilities still generate a lot of waste heat, and Europe is ahead of the United States in discovering ways to harness it, such as heating pools, Serter said.

“We still think demand will outpace the efficiency gains we're seeing,” said Bruce Owen, UK managing director of data center company Equinix. He predicts more data centers will be built with on-site power generation facilities. Equinix was refused planning permission for a gas-fueled data center in Dublin last year.

Sartor adds that cost may ultimately determine whether generative AI is worthwhile for a particular application: “If the old way is cheaper and easier, there's not much of a market for the new way.”

However, Dr. Luccioni emphasizes that people need to clearly understand how the options before them differ in terms of energy efficiency. She is working on a project to develop her AI energy assessment.

“Instead of choosing this GPT derivative model, which is very cumbersome and consumes a lot of energy, you can choose this A+ Energy Star model, which is much lighter and more efficient,” she says.