GPT-4, the latest version of the model that powers ChatGPT, analyzes data with perhaps a trillion parameters, more than five times as many as previous versions. As models become more complex, the computational need to train them increases accordingly.
AI, once trained, does not require much computing power to use in a process called inference. However, given the range of applications offered, inference also cumulatively demands a lot of processing power.
Microsoft has more than 2,500 customers with services that use technology from ChatGPT creator OpenAI, nearly half of which are owned by software giants. This is a tenfold increase compared to the previous quarter. Google’s parent company, Alphabet, has six of his products with more than 2 billion users worldwide and plans to enhance these products with generative AI.
clear winner
The most obvious winners from the surge in demand for computing power are chip makers. Companies such as Nvidia and AMD receive licensing fees whenever their blueprints are etched into silicon by manufacturers such as TSMC on behalf of their end customers, especially the cloud computing giants that power most AI applications. increase. As such, AI benefits from more powerful chips (which tend to yield higher margins) and more chips, so it’s a boon for chip designers.
Bank UBS expects AI to boost demand for specialty chips known as graphics processing units (GPUs) by $10 billion to $15 billion in the next year or two.
As a result, Nvidia’s annual data center revenue, which accounts for 56% of sales, could double. AMD is expected to launch new GPUs later this year. The company is a much smaller player than his Nvidia in the GPU design game, but the scale of the AI boom means the company is poised to profit “even if it’s just the wreckage” of the market. Bernstein’s Stacey Rasgon said: broker.
AI-focused chip design startups such as Cerebras and Graphcore are trying to make a name for themselves. Data provider Pitchbook counts about 300 such companies.
Naturally, some of the inventory also accrues to manufacturers. In April, TSMC boss CC Wei was cautious about the “gradual upside of AI-related demand.” Investor interest is growing. The company’s stock price jumped 10% after Nvidia’s latest results, adding about $20 billion to its market capitalization.
Less obviously, companies that can package more chips into a single processing unit are also benefiting. Besi is a Dutch company that makes tools that help join chips. Dutch firms control three-quarters of the precision adhesive market, according to Pierre Ferrag of another analyst firm, New Street Research. The company’s stock has risen more than half this year.
UBS estimates that GPUs account for about half the cost of specialized AI servers, compared to one-tenth the cost of standard servers. But that’s not the only gear you need. GPUs in a data center also need to communicate with each other to operate as a single computer.
This requires increasingly sophisticated network equipment such as switches, routers and specialized chips. The market for such kits is expected to grow 40% annually over the next few years, reaching nearly $9 billion by 2027, according to research firm 650 Group. Nvidia also licenses these devices, which account for 78% of global sales.
But competitors like California firm Arista Networks have also caught the eye of investors, with the company’s shares up nearly 70 percent over the past year. Broadcom, which sells specialized chips that help networks operate, said annual sales of such chips will quadruple to $800 million by 2023.
The AI boom is also good news for companies building servers in data centers, said Peter Rutten of IDC, another research firm. Already he’s one analyst firm, the Dell’Oro Group, whose share of AI-dedicated servers in data centers around the world has gone from less than 10% today to about 20% in five years. increase, and data center capital expenditures are forecast to increase. The percentage on the server will increase from about 20 percent today to 45 percent.
This will benefit server makers such as Taiwan’s Wistron and Inventec, which primarily produce custom-built servers for giant cloud providers such as Amazon Web Services (AWS) and Microsoft’s Azure. Small manufacturers should do well too.
The management of Wiwynn, another Taiwanese server maker, recently said that AI-related projects account for more than half of its current orders. US company Supermicro said AI products accounted for 29% of its sales in the three months to April, up from an average of 20% over the past 12 months.
specialized software
All this AI hardware requires specialized software to work. Some of these programs are provided by hardware companies. For example, Nvidia’s software platform called CUDA allows a customer to get the most out of his GPU.
Other companies are creating applications that allow AI companies to manage data (Datagen, Pinecone, Scale AI) and host large language models (HuggingFace, Replicate). PitchBook counts about 80 such startups. More than 20 companies have raised new funding so far this year. Pinecorn counts two big names in venture capital, Andreessen Horowitz and Tiger Global, as investors.
As with hardware, many of the main customers for this software are cloud giants. Together, Amazon, Alphabet and Microsoft plan to spend about $120 billion in capital spending this year, up from $78 billion in 2022. Most of that goes to expanding cloud capacity. Yet the demand for AI computing is so high that even they are struggling to keep up.
That paved the way for challengers. In the last few years, IBM, Nvidia, and Equinix have started offering access to his GPU “as a service.” AI-focused cloud startups are also proliferating. One of them, Lambda, raised $44 million in March from investors including Gradient Ventures, one of Google’s venture arms, and OpenAI co-founder Greg Brockman. The deal valued the company at about $200 million.
A similar company, CoreWeave, raised $221 million in April, including funding from Nvidia, at a $2 billion valuation. CoreWeave co-founder Brannin McBee argues that a focus on customer service and infrastructure designed around AI will help the company compete with cloud giants.
The final group of AI infrastructure winners, who are closest to providing a real shovel, are data center landlords. That property is filling up as the demand for cloud computing surges. Data center vacancy in the second half of 2022 will hit a record low of 3%.
Specialty firms such as Equinix and rival Digital Realty are increasingly competing with big asset managers who want to add data centers to their real estate portfolios. In 2021, private market giant Blackstone paid $10 billion to QTS Realty Trust, one of America’s largest data center operators. Blackstone’s Canadian rival Brookfield, which has invested heavily in data centers, acquired French data center firm Data4 in April.
potential constraints
As AI infrastructure stacks continue to grow, they may face constraints. One is energy. Big investors in data centers say the sheer amount of electricity they draw is expected to slow the development of new data centers in locations like Northern Virginia and Silicon Valley. there is
Another potential block is moving from vast AI models and cloud-based inference to training that requires less computing power to run inference on smartphones, as in the case of a scaled-down version of Google’s recently announced Palm model. It is a migration to a smaller system that can .
The biggest question mark rests on the permanence of the AI boom itself. Despite the popularity of ChatGPT and its likes, the beneficial use cases for this technology remain obscure. In Silicon Valley, hype can suddenly turn into disappointment. Nvidia’s market value skyrocketed in 2021 as the company’s GPUs proved to be perfect for mining Bitcoin and other cryptocurrencies, but fell as the crypto boom collapsed.
And if the technology lives up to its transformative claims, it could be cracked down by regulators. Policy makers around the world, concerned about the potential for generative AI to kill jobs and spread misinformation, are already considering guardrails. Indeed, on May 11, EU lawmakers proposed a set of rules to limit chatbots.
All of these limiting factors could slow AI adoption, and in doing so could weaken the prospects for AI infrastructure companies. But maybe just a little. Even if generative AI doesn’t turn out to be as revolutionary as its boosters claim, it will almost certainly be more useful than cryptocurrency.
There are many other non-generative AIs that require a lot of computing power. A global ban on generative AI is the only way to stop the gold rush, but that is not imminent. And as long as everyone is in a hurry, pickaxe and shovel hawkers will make money.
