- After training on 100,000 Nvidia H100 GPUs, Elon Musk said Grok 3 will be “something special.”
- Nvidia's H100 GPUs, a key component of the AI, are estimated to cost between $30,000 and $40,000 each.
- It's unclear how many H100s Musk's company has purchased, but GPU computing is also available for rent.
Elon Musk is heavily hyping the next version of his AI chatbot, Grok.
The billionaire responded to X's post on Monday, saying that the latest version of xAI's chatbot, Grok 3, should be “something special” once it's trained on 100,000 H100s.
Musk was referring to NVIDIA's H100 graphics processing unit, also known as Hopper, a cutting-edge technology in AI that helps process data for large language models (LLMs). The chip is a key component in AI development and has become a hot commodity in Silicon Valley as tech companies race to build ever-smarter AI products.
Each Nvidia H100 Graphics Processor Tips Estimated Cost It's about $30,000, Quote Costs can run up to $40,000. Quantity discounts may be available.
Based on these estimates, Grok 3 is being trained on AI chips worth $3 billion to $4 billion, though it's unclear if those chips were purchased outright by Musk's company. It's also possible to rent GPU computing from cloud service providers, and The Information reported in May that Musk's xAI startup was in talks with Oracle to spend $10 billion over multiyear to rent cloud servers.
However, we know that Musk's company has directly purchased large quantities of H100s in recent years – for example, the Tesla CEO reportedly diverted $500 million worth of Nvidia H100s that were destined for Tesla into the X.
Training based on 100,000 GPUs would be a big step up from Grok 2. In an April interview with Norwegian sovereign wealth fund chief Nikolai Tangen, Musk said that Grok 2 training would require around 20,000 H100s.
xAI has previously released Grok-1 and Grok-1.5, with the latest version only available to early testers and existing users of X (formerly Twitter). In an X post on Monday, Musk said Grok 2 was due for release in August, while in another post about the GPU he hinted that Grok 3 would be released at the end of the year.
xAI did not respond to a request for comment.
100,000 GPUs sounds like a lot, and it is. But other big tech companies Meta Meta is piling on more GPUs: Mark Zuckerberg said in January that Meta plans to buy about 350,000 Nvidia H100 GPUs by the end of 2024. He also said that Meta plans to have about 600,000 chips, including other GPUs.
If that's the case, Meta would have spent roughly $18 billion building out its AI capabilities.
The stockpiling of H100 chips also contributed to how ruthless recruiting for top AI talent became last year.
Aravind Srinivas, founder and CEO of AI startup Perplexity, said he was turned down by researchers at Meta AI when they tried to recruit Zuckerberg because of his vast collection of AI chips.
“I tried to hire some very senior researchers from Meta, and guess what they said? 'Come back when you have 10,000 H100 GPUs,'” Srinivas said.
