GPUs (graphics processing units) are special chips originally designed to render graphics in video games, and have since become the cornerstone of the artificial intelligence arms race. These are expensive and rare, mostly from his Nvidia Corp, whose market value topped $1 trillion last month due to a surge in demand. To build AI models, developers typically purchase access to their servers in the cloud from companies such as Microsoft Corp. and Amazon.com Inc. These servers are equipped with GPUs.
There is a saying that sells shovels during the gold rush. It’s no surprise that today’s AI infrastructure providers are profiting. But there’s a big difference now from his mid-19th century, when the winners of the California gold rush were upstarts like Levi Strauss and Samuel Brennan in durable miner’s trousers. , sold enough pots to become a millionaire. Now, and at least for the next year or so, most of the profits from selling AI services will go to companies like Microsoft, Amazon, and Nvidia that have already dominated the tech space for years.
One reason is that the cost of cloud services and chips is rising, while the price of accessing AI models is falling. In September 2022, OpenAI cut his GPT-3 access cost by a third. Six months later, access is ten times cheaper. And in June, OpenAI reduced fees by 75% for embedded models that convert words to numbers so large language models can process context. Information costs are “on the road to near zero,” said Sam Altman.
Meanwhile, buying a GPU today is like trying to buy toilet paper during the COVID-19 pandemic, so the price of building AI models is rising. Nvidia’s A100 and H100 chips are the gold standard for machine learning computing, but the price of the H100 has risen from under $35,000 just a few months ago to over $40,000, and a global shortage has forced Nvidia to stock up on chips. It means that it cannot be manufactured at high speed. Many AI startups have found themselves waiting in line behind big customers like Microsoft and Oracle to buy these coveted microprocessors. A Silicon Valley-based startup founder with ties to Nvidia said even OpenAI is waiting for the H100 chip and won’t get it until spring 2024. A spokeswoman for OpenAI said the company has not made that information public. But Altman himself is having trouble getting chips, he complains.
Big tech companies have a huge advantage over start-ups like OpenAI, not only because of their established customer base, but also because they have direct access to all-important GPUs. Until Sam Altman thinks that dabbling in big cloud vendors may be the safest way for AI companies to stay in business when he trades 49% of OpenAI for Microsoft’s $1 billion investment in 2022. , seemed like a lot of stock to part with. .
So far, the bet has paid off for Microsoft. The company’s chief financial officer, Amy Hood, told investors in June that the AI-powered services the company sells, including OpenAI, contribute at least $10 billion to the company’s revenue. . She called it “the fastest growing $10 billion business in our history.” Its Microsoft product, called Azure OpenAI, is more expensive than OpenAI’s own product, but companies like CarMax and Nota have checked in on security and compliance issues, etc., to access GPT-4 in a more enterprise-friendly way. It can be so.
On the other hand, AI model makers face a constant movement of talent between companies, making it difficult to maintain confidentiality and product differentiation. And the costs are endless. Once he spends his money on cloud credits to train his models, he also needs to run those models for his customers. This is a process known as inference. AWS estimates that inference accounts for up to 90% of the total operating costs of AI models. Most of that money goes to cloud providers.
This sets the stage for a two-tier system for AI business. Those at the top have money and prestigious connections. Founders graduating from elite startup accelerator Y Combinator are offered computing credits worth hundreds of thousands of dollars from cloud vendors such as Amazon and Microsoft. A lucky few have managed to work with venture capital investor Nat Friedman. He recently spent an estimated $80 million on his own batch of GPUs to set up a bespoke cloud service called the Andromeda Cluster.
Tier 2 AI companies will be the long tail companies that don’t have these kinds of connections and resources to train AI systems, no matter how smart their algorithms are.
The silver lining for small businesses is that big tech companies will one day be forced to commoditize their products and services and loosen the squeeze on the market for building AI. The chip shortage will eventually ease, making GPUs more accessible and cheaper. Competition should also intensify as cloud providers encroach on each other’s territory. For example, Google has developed its own version of the GPU, called TPU, and Nvidia is building its own cloud business to compete with Microsoft.
Also, as researchers develop techniques such as LoRA(1) and PEFT(2) to make the AI model building process more efficient, the required data and computational power will decrease. AI models are currently moving towards miniaturization. That way less GPUs and infrastructure are needed. This means that Big Tech’s dominance will not last forever.
Details from Bloomberg Opinion:
• IBM will not be among the top ranks of the tech industry: Dave Lee
• AI and cryptocurrencies are becoming regulatory enemies: Aaron Brown
• Ghosts in machines should not be AI: Parmie Olson
(1) Low-rank adaptation of large language models is a training method that speeds up the training of large models while keeping memory consumption low.
(2) Parameter-efficient fine-tuning enables AI model authors to achieve performance comparable to full model fine-tuning, even with a small number of trainable parameters.
This column does not necessarily reflect the opinions of the editorial board or Bloomberg LP and its owners.
Palmy Olson is a technology columnist for Bloomberg Opinion. She is a former Wall Street Journal and Forbes reporter and author of We Are Anonymous.
More articles like this can be found at bloomberg.com/opinion.
