Many parts of the world, including the leading US technology hubs, are eager to see AI factories come online, pending the creation of new energy infrastructures to power them.
Washington, DC-based startup Emerald AI is developing AI solutions that allow next-generation data centers to come online faster by tapping existing energy resources in a more flexible and strategic way.
“Traditionally, power grids have treated data centers as inflexible. Energy system operators assume that a 500 megawatt AI factory will always need access to that full amount of electricity.” “However, if grid peaks and supply demand are short at the moment needed, the workloads that promote energy use in AI factories will be flexible.”
That flexibility is enabled by the startup's emerald conductor platform. This is an AI-equipped system that acts as a smart mediator between the grid and the data center. In a recent field test in Phoenix, Arizona, the company and its partners demonstrated that its software can reduce the power consumption of AI workloads running on clusters of 256 NVIDIA GPUs during grid stress events by 25% in three hours while maintaining the quality of computing service.
Emerald AI achieved this by tuning hosts for the various workloads that the AI factory runs. Some jobs can be paused or slowed, such as training and tweaking large-scale language models for academic research. Others cannot reschedule, like inference queries for AI services used by thousands or millions, but can redirect to another data center with less stress on the local power grid.
Emerald conductors coordinate these AI workloads across the data center network to meet power grid demands and dynamically reduce flexible workload throughput within acceptable limits while ensuring full performance of time-sensitive workloads.
Not only will it help AI factories get online using existing power systems, but its ability to adjust power usage will help cities avoid blackouts, protect communities from rising utility rates, and make it easier for the grid to integrate clean energy.
“We're a great fan of our efforts to help people understand how we're doing,” said Ayse Coskun, chief scientist at Emerald AI and professor at Boston University. “The data center could become part of these shock absorbers.”
Emerald AI, a member of the NVIDIA Inception program at the Startups for Nventures Portfolio Company, announced today more than $24 million in seed funding. The Phoenix demonstration, part of EPRI's DCFLEX data center flexibility initiative, was carried out in collaboration with NVIDIA, Oracle Cloud Infrastructure (OCI), and the Regional Power Utility Salt River Project (SRP).
“The Phoenix Technology Trial examines the great potential of key factors in data center flexibility,” said Anuja Ratnayake, head of EPRI's DCFlex consortium.
EPRI also leads the Open Power AI Consortium, a group of energy companies, researchers and technology companies working on AI applications in the energy sector.
Make the most of your grid
Electric grid capacity is not normally used unless there is a high power demand for cooling and heating, such as peak events such as hot summer days or cold winter storms. This means that in many cases, there is space in the existing grid for new data centers, as long as energy usage can be temporarily dialed down during peak demand periods.
A recent Duke University study estimates that if a new AI data center can only increase its power consumption of less than 200 hours a year, it will be able to unlock a new 100 gigawatt capacity to connect the data center.
Test the flexibility of AI Factory
Recent exams on emerald AI took place in the Oracle Cloud Phoenix region of Nvidia GPUs spreading across multi-rack clusters managed through DataBricks MosaicMl.
“The rapid delivery of high-performance calculations to AI customers is critical, but it is constrained by grid power availability,” said Pradeep Vincent, Chief Technical Architect and Senior Vice President of Oracle Cloud Infrastructure, which provided the trial's cluster power telemetry. “Complete performance while calculating infrastructure that corresponds to real-time grid conditions unlocks new models to scale AI.
Jonathan Frankle, chief AI scientist at Databricks, has led the AI workload testing choice and flexibility thresholds.
“There is some potential flexibility in how AI workloads typically run,” Frankl said. “In many cases, a small portion of a job can't really be amortized, but many jobs, such as training, batch inference, and fine-tuning, have different priorities levels depending on the user.”
As Arizona is one of the top states for data center growth, SRP sets a flexibility target for AI computing clusters (25% power consumption reduction compared to baseline load) to demonstrate how new data centers can provide meaningful relief to Phoenix's power grid constraints.
“This test was an opportunity to completely rethink AI data centers as a resource that would help them work perfectly and reliably,” said David Roussea, president of SRP.
On May 3rd, when air conditioning demand was high, a hot day in Phoenix, SRP's system experienced peak demand at 6pm during testing. The data center cluster gradually reduced consumption on a 15-minute ramp, maintaining a 25% power cut over three hours, rising without exceeding the original baseline consumption.
AI Factory users can label workloads and guide emerald software that can slow, pause, and schedule jobs. Alternatively, an emerald AI agent can automatically create these predictions.
Orchestration decisions were guided by an emerald simulator. This accurately modeles the behavior of your system and optimizes the trade-off between energy usage and AI performance. Demand for historical grids from data provider Amperon confirmed that AI clusters functioned correctly during peak periods of grids.
Building an energy-resistant future
The International Energy Agency predicts that by 2030, electricity demand from data centers could more than double. In light of the expected demand for grids, Texas has passed legislation requiring consumption from the grid to be reduced or cut off in utility requests during roadshed events.
“In these circumstances, if data centers can dynamically reduce energy consumption, they may be able to avoid completely stopping power sources,” Shivalam said.
In the future, Emerald AI is expanding its technology trials from Arizona onwards. We plan to continue working with NVIDIA to test technology in our AI factories.
“You can make your data centers controllable while ensuring acceptable AI performance,” says Sivaram. “AI factories can succumb when the grid is tight. Users will sprint when they need it.”
Learn more about starting NVIDIA. Explore AI platforms designed for power and utilities.
