GPU prices have soared this year, highlighting how far the AI chip market is from post-boom normalcy.
Carmen Li, CEO of Silicon Data, said in an interview that the company’s pricing data shows significant price increases across old and new Nvidia GPUs used to train and run AI models.
The company’s Neo Cloud H100 index rose from 2.20 to 2.64, an increase of 20% in the past three months. The Neo Cloud B200 index rose from 4.40 to 5.35, an increase of 22%. Also, the hyperscalar H100 index rose 3% from 7.26 to 7.46, according to Silicon Data.
This number suggests that demand for AI computing still exceeds supply. Lee said the market is not following the usual pattern of rising prices when new chips are first released and then gradually falling as supply improves. Instead, B200 prices have remained elevated and are rising further, a sign that capacity constraints remain acute, she said.
Lee argued that pressures from this supply-demand imbalance are being felt across the AI stack, including chips, memory, power, and data center space. This constraint keeps prices strong even on older hardware such as Nvidia’s H100, which is still central to AI training and inference workloads.
“Prices have gone up quite a bit,” Lee said in a recent interview about H100 rental prices.
Cloud pricing power
The premium on GPU rental prices is greatest in hyperscaler environments, where customers pay more for convenience, pre-existing integration, and access to capacity from the world’s largest cloud providers.
According to Silicon Data, the H100 costs nearly three times as much to rent from hyperscalers like Amazon, Microsoft, Google, and Oracle as it does from neocloud providers like CoreWeave that run specialized AI services.
Mr. Lee spent more than two years as global head of strategic alliances at financial data giant Bloomberg, where he oversaw enterprise data transactions. She retired in 2024 and started Silicon Data specifically to track GPU prices. She aims to increase transparency in the GPU rental market. Lee said the company captures hundreds of thousands of pricing points around the world and normalizes them into an index. This can be used to understand both spot rental and long-term value.
Not much depreciation
This pricing information is critical as tech giants, AI labs, and other companies spend trillions of dollars buying GPUs and building data centers to fuel the generative AI boom. One concern is that if GPU prices fall due to weak demand, the AI cloud giant could have to depreciate these assets faster, hurting its revenue.
So far, the opposite has happened. Li, who also runs a business called Compute Exchange, said GPU prices have been going up and even older GPUs haven’t come down much in price. This is a company that resells refurbished GPUs.
In the second year, Lee said he could sell refurbished H100s for 85 cents on the dollar. And in the third year, you can sell the same GPU for 84 cents on the dollar.
“My car depreciates much faster than that,” she said.
Currently, the demand for AI is unexpected and the supply of computing cannot keep up. This has implications not only for infrastructure budgets, but also for the economics of AI products that rely on rented GPUs to generate AI tokens at scale.
Sign up for BI’s Tech Memo newsletter here. Please contact us by email. abarr@businessinsider.com.