Investors who poured money into SK Hynix’s $28 billion blockbuster Nasdaq debut should be aware that the business model that the world’s leading memory chip makers thrive on is about to shift to one that requires a bit more strategic and financial gamble.
So far, SK Hynix has made all the right moves, but it’s unclear how well the sector’s leading players will cope with the disruption.
The trigger, of course, is the artificial intelligence (AI) boom. More powerful AI systems are pushing memory chips to their limits: transferring data to and from processors, the brains of computers and servers.
For most of the history of computing, the calculations on a processor chip were the slowest part.
The situation has reversed, as today’s graphics processing units, made by Nvidia Corp., can perform calculations much faster, while data sent and received from memory chips is still slow by comparison.
Imagine a genius scholar who can’t write things down fast enough because his notepad has become a memory chip.
“AI machines have very large models, so they don’t have enough memory bandwidth,” said Pat Gelsinger, former president of Intel.
This means that the AI chip is not reaching its full potential, and even the most sophisticated chatbot has a limit to how much it can remember.
For example, Anthropic’s Fable 5 can only hold about seven novels’ worth of text in its working memory.
If you ask for further consideration, you should start removing previous ones.
AI builders have been trying for years to force their models to remember much more, allowing them to perform more sophisticated and continuous tasks in areas such as finance and corporate back-office operations.
But they are partially constrained by physics. Servers, the computers in data centers that run AI systems, use a technology called high-bandwidth memory (HBM).
The idea is to stack the basic components of a memory chip on top of each other to reduce the distance data travels to and from the processor (GPU). However, this technology has flaws.
“HBM as we know and love it today is a terrible memory. Heat is trapped in the lower layers of the memory chip, and the increased temperature forces the GPU to throttle its speed,” he said.
One answer is that the world’s three largest memory chip makers – SK Hynix, Samsung Electronics Co and Micron Technology Inc – are building memory directly into the processor itself.
Instead of a genius scholar having to write something down, she effectively has a notepad in her head.
That would make AI models many times more powerful, but it also means the three memory chip makers, whose combined market value has increased by $3 trillion over the past year thanks to strong demand, will have to change the way they do business.
An industry that has always produced standardized chips that can be sold like a commodity now has to produce custom products.
“This is a completely new business model for them,” Gelsinger said. As a general partner at venture capital firm Playground Global, Gelsinger invests in startups focused on integrating memory and computing power into the same silicon.
Because it takes years to design and mass-produce new custom chips, he expects at least two to three more years of good times for the big three before disruption begins in earnest.
Nvidia has responded to upcoming changes by signing a US$20 billion licensing deal with California-based Groq Inc, which puts high-speed memory on its processors.
This poses some kind of threat to SK Hynix and its peers, which also need to get into the business of predicting future sales for Nvidia, Intel, Alphabet Inc.’s Google and other companies that make advanced processors, and spend appropriate amounts on R&D and chip manufacturing to meet demand.
In other words, it’s no longer about building as many of the same memory chips as possible, but about making the right bets about who will win and who will lose in the market for selling computer brains.
If you choose right, you’ll have the perk of locking in customers for years. If you make the wrong choice, you could be left with a pile of expensive chips.
SK Hynix chose to hedge its 2024 risk by partnering with Taiwan Semiconductor Manufacturing, which makes chips for nearly every major tech company.
This would allow South Korean memory companies to sell to competing tech giants rather than betting on one company.
So far, it’s doing well, riding on market enthusiasm.
As the AI trade moves to other areas of infrastructure construction, the memory boom has been so strong that investors have dumped NVIDIA stock in favor of SK Hynix and its peers. (SK Hynix stock has increased 600% in the past 12 months.)
This is a surprising change for a commoditized market that has looked rather boring for the past 50 years. “Never in history has memory been so exciting,” Gelsinger says.
For companies and investors who are constantly entering the market, exciting may be another word for risk. — Bloomberg
Palmy Olson is a technology columnist for Bloomberg Opinion. The views expressed here are the author’s own.
