In 2022, US chipmaker Nvidia will release the H100, one of the most powerful processors the company has ever developed. He is also one of the most expensive processors at around $40,000 per unit. The launch seemed ill-timed at a time when companies were trying to cut spending amid rampant inflation.
ChatGPT launched in November.
“Last year was pretty tough, but it turned around overnight,” said Jensen Huang, CEO of Nvidia. “I see,” he said, of OpenAI’s chatbot hit. “It created an immediate demand.”
ChatGPT’s sudden popularity has sparked an arms race among the world’s leading tech companies and startups, rushing to acquire what Huang describes as “the world’s first computer,” the H100. [chip] “Designed for Generative AI” is an artificial intelligence system that can rapidly create human-like text, images, and content.
This week, the value of getting the right product at the right time became clear. Nvidia said on Wednesday that a resurgence in data center spending by big tech and demand for its AI chips would bring in $11 billion in revenue in the three months to July, up from earlier Wall Street estimates. more than 50%.
Investor reactions to the forecast added NVIDIA’s market capitalization to $184 billion in a single day on Thursday, pushing the valuation of the already world’s most valuable chip company closer to $1 trillion.
Nvidia is an early winner in the astronomical rise of generative AI, a technology that threatens to reshape industries, boost productivity and take millions of jobs.
That technological leap would be fueled by the H100, which is based on a new Nvidia chip architecture called “Hopper,” named after American programming pioneer Grace Hopper, and suddenly became the most popular in Silicon Valley. It has become a product with
“All of this started right when we started production on Hopper,” Huang said, adding that large-scale production began just weeks before ChatGPT debuted.
Huang’s confidence in continued profitability led him to work with chipmaker TSMC to produce the H100 to meet explosive demand from cloud providers such as Microsoft, Amazon and Google, Internet groups such as Meta, and enterprise customers. This is due in part to the ability to scale the
“This is one of the scarcest engineering resources on the planet,” said chief strategy officer of AI-focused cloud infrastructure startup CoreWeave, one of the first to ship the H100 earlier this year. Maker and Founder, Branin McBee said.
Some customers have waited up to six months to get the thousands of H100 chips needed to train their massive data models. The AI startup had expressed concern that the H100 would be in short supply just as demand began to rise.
A snapshot of an interactive graphic is shown. This is most likely due to being offline or having JavaScript disabled in your browser.
Elon Musk, who bought thousands of Nvidia chips for his new AI startup X.ai, told a Wall Street Journal event this week that GPUs (graphics processing units) are currently “more than drugs. It’s very difficult to get,” he said. It was “not really a high hurdle in San Francisco,” he joked.
“Computing costs are astronomical,” Musk added. “server he minimum investment in hardware he must be $250 million” [to build generative AI systems]”
The H100 promises higher performance, so big tech companies like Microsoft and Amazon that are building entire data centers around AI workloads, and generative AI startups like OpenAI, Anthropic, Stability AI, and Inflection AI. It has proven to be particularly popular among businesses. This can accelerate product launches and reduce training costs over time.
“In terms of gaining access, it’s certainly like launching a GPU with a new architecture,” said Ian Buck, head of Nvidia’s hyperscale and high-performance computing business. We are tackling the difficult task of increasing the supply of H100 to “It’s happening at a super-scale,” he added, noting that some big customers are asking for tens of thousands of GPUs.
An “accelerator” designed to run in data centers, the unusually large chip contains 80 billion transistors, five times the processor in the latest iPhone. It costs twice as much as its predecessor, the A100, which launched in 2020, but early adopters say the H100 boasts at least three times the performance.
“The H100 solves the scalability problem that has plagued us in the past. [AI] Modeler,” said Emad Mostaque, co-founder and CEO of Stability AI, one of the companies offering Stable Diffusion image generation services. “This is important because it allows us to train larger models faster as we move from research to engineering problems.”
The timing of the H100’s launch was ideal, but Nvidia’s breakthrough in AI can be traced almost two decades back to innovation in software, not silicon.
Created in 2006, the company’s Cuda software allows GPUs to be repurposed as accelerators for other kinds of workloads beyond graphics. And he explained that around 2012, Buck said, “AI found us.”
Canadian researchers have found GPUs to be ideal for creating neural networks, a type of AI inspired by the interaction of neurons in the human brain. Neural networks were becoming the new focus of AI development at the time. “It took us almost 20 years to get to where we are today,” Buck says.
Nvidia now has more software engineers than hardware engineers, which will help support the various kinds of AI frameworks that will emerge in the years to come, and will make the chips more efficient in the statistical computations needed to train AI models. You can increase your efficiency.
Hopper is the first architecture optimized for “transformers,” the approach to AI that underpins OpenAI’s “generative pre-trained transformer” chatbots. His close collaboration with AI researchers allowed Nvidia to spot the emergence of Transformers in 2017 and start adjusting its software accordingly.
“NVIDIA was probably the first to see the future with its focus on making GPUs programmable,” said Nathan Benaich, general partner at AI startup investor Air Street Capital. says. “The company saw opportunities, gambled big, and consistently outperformed its competitors.”
Benaich estimates that NVIDIA has a two-year lead over its rivals, but adds that “its position is by no means unshakable, both in terms of hardware and software.”
Stability AI’s Mostaque agrees. “Google, Intel, and other next-generation chips are catching up.” [and] Even Cuda has lost its moat as software has become more standardized. “
For some in the AI industry, this week’s Wall Street enthusiasm seems overly optimistic. Still, “for the foreseeable future,” “the AI market for semiconductors will continue to be dominated by NVIDIA’s winner,” said Jay Goldberg, founder of chip consulting firm D2D Advisory.
Additional report by Madhumita Murgia
