Nvidia CEO Jensen Huang announced the new inference system at the company’s annual GTC conference on Monday. This is the company’s most decisive move yet to protect its advantage as inference becomes AI’s next battleground.
The new push into inference comes after Huang said Nvidia is anticipating huge demand. The company expects demand for its Blackwell and Rubin AI systems to be at least $1 trillion by 2027, compared with projected demand of about $500 billion by 2026, he said.
The AI chip giant has announced the new Nvidia Groq 3 LPX. This can speed up inference workloads by up to 35 times, Huang said. It will integrate technology from AI chip startup Groq and combine it with Nvidia’s Vera Rubin architecture.
Samsung is manufacturing new Groq chips, and Nvidia expects the system to ship later this year.
“We have reached an inflection point in reasoning,” Huang said in his keynote address.
Nvidia’s new system builds on a roughly $20 billion deal it signed with Groq in December to license Groq’s technology and hire top engineers.
Huang previously hinted at a partnership with startup Groq during Nvidia’s latest earnings call. The Wall Street Journal previously reported that the company is preparing a new inference system that incorporates Groq technology.
Nvidia’s graphics processing units (GPUs) continue to dominate the AI field and can be used for both training and inferring AI models, or how AI models make decisions and predictions.
Now, a growing number of Nvidia competitors, from hyperscalers to chip startups, are developing cheaper, more efficient, purpose-built systems for repetitive, cost-sensitive inference tasks.
The rise of AI agents, tools that perform tasks on behalf of humans, could dramatically increase the demand for inference.
To this end, AI companies like OpenAI have been exploring alternatives to Nvidia hardware. Reuters previously reported that there were complaints about the company’s inference chips. In January, OpenAI signed a reported $10 billion computing deal with inference chip startup Cerebras.
Any tips? To contact this reporter via email, please specify the following address: gweiss@businessinsider.com Or send a signal at @geoffweiss.25. Use a personal email address, non-work WiFi network, and non-work device. here‘is our guide Share information securely.
