Taipei, December 1, 2025 /PRNewswire/ — The rapid expansion of artificial intelligence (AI) workloads is forcing a fundamental redesign of data center infrastructure, creating historic memory shortages and opening a rare market window for alternative memory technologies, according to Chuck Sobey, Founder of ChannelScience and General Chair of Chiplet Summit.
Speaking about changes in the industry, Sobey emphasized that AI workloads are structurally different from traditional enterprise tasks. While traditional workloads are CPU, network, and storage intensive, often forcing processors to wait for I/O, modern AI tasks such as deep learning training, large-scale language model (LLM) inference, and search augmentation generation (RAG) are critically limited by memory bandwidth.
Bandwidth gap and power crisis
The demands of the AI era are unprecedented. Sobey noted that AI systems can require up to 10 terabytes per second (TB/s) of GPU memory access, which is roughly 50 to 200 times the bandwidth of standard DDR5. Unlike web and cloud requests, which are processed in kilobytes, AI access involves gigabyte-sized tensors, requiring algorithms that run through the entire memory storage hierarchy.
This architectural burden has significant economic implications. Sobey described the modern AI data center:token In “Factory”, revenue is derived strictly from the processing of tokens. Currently, these facilities are limited by power and memory rather than computing limitations. Power density requirements have skyrocketed, requiring rack redesigns to support up to 1 megawatt, exceeding previous standards by a factor of 10 to 100.
Supply chains are already straining under this pressure. Sobey pointed to historic shortages in the memory sector, particularly high-performance server memory, with some reports of order fulfillment rates reaching 70% and price increases of up to 50%. Obtaining the necessary memory inventory is becoming increasingly difficult for all but the largest hyperscalers.
Chiplets: Gateway to MRAM and RRAM
Sobey identified this supply/power shortage as a strategic entry point for “alternative memory,” technologies such as MRAM (magnetoresistive RAM), RRAM (resistive RAM), and phase change memory (PCM) that do not have the incumbent companies’ multibillion-dollar dedicated fabs.
Chiplet architecture is a key enabler of this transition. By breaking down the functionality of monolithic ASICs into smaller units, chiplets allow designers to integrate the esoteric materials needed for alternative memories without contaminating high-end logic manufacturing processes. This heterogeneous integration enables certain performance advantages such as radiation resistance and high temperature tolerance, which Sobey argues are necessary differentiators beyond simple cost claims.
New “5 second rule”
The speed of AI is also compressing economic rules of thumb. Sobey noticed that the long-standing “5-minute rule,” which states that data should be stored in DRAM if it is accessed within 5 minutes, has accelerated to the “5-second rule.” In the AI era, if data is not accessed within 5 seconds, cost-benefit analysis requires it to be evicted from expensive and fast memory tiers.
Sobey said the current AI surge is more than just a ripple, it’s a collection of waves, and argued that suppliers need to secure their position now if they want to maintain an advantage when pricing and supply chains eventually settle down.
Sobey will discuss the impact of chiplets on memory and storage hierarchies in more detail during his talk. DIGITIMES Webinar December 5, 2025, and chiplet summit Scheduled for February 17-19, 2026 in Santa Clara, California.
Source DIGITIMES ASIA
