Augmented by artificial intelligence
Getty
This post focuses on two recent product announcements using CXL for accelerated artificial intelligence (AI) workloads. These include Korean Panmesia’s TrainingCXL PCIe device and Enfabrica’s high-performance CXL switch chip that accelerates a variety of AI applications.
Panmesia, a startup spun out from Korea Advanced Institute of Science and Technology (KAIST), has introduced an AI acceleration device using Compute Express Link (CXL) technology (TrainingCXL). This device connects the GPU to multiple CXL memory expanders. The system can scale up to 4PB of memory capacity in a fine-grained memory pool and, according to the company, can reduce training run times by 5.3x compared to leading systems using existing PCIe interconnects. Below is an image of a PCIe based TrainingCXL device.
TrainingCXL device image
Images from the TrainingCXL presentation
TrainingCXL features a configurable architecture that connects a large new memory-loaded memory expander device to the GPU via CXL, providing a scalable memory space for the GPU. It also minimizes the overhead of moving data between connected devices by employing the cxl.cache subprotocol, one of several subprotocols provided by CXL. This sub-protocol enables fast, active data transfers between GPUs and memory expanders without software intervention. This reduces CPU overhead and improves overall performance.
In addition to performance acceleration, TrainingCXL includes domain-specific computing (accelerator technology) for deep learning recommendation models. According to the company, current recommendation systems often require tens of terabytes of memory to store information about each user and content. This information is stored in what is called an embedding vector.
TrainingCXL stores the embedding vectors in a memory expander and has a compute module that processes them directly in the memory expander. This reduces the load on the GPU, speeds up the processing of embedding vectors, and sends a reduced data set to the GPU instead of a large embedding vector, further improving performance.
The work of the Panmnesia research team was published in the March-April 2023 issue of IEEE Micro magazine in an article titled “Fail-tolerant training with persistent memory decomposition using CXL.” He was also an invited speaker at his HCM workshop co-located with his IEEE International Symposium on High Performance Computer Architecture (HPCA) in Montreal, Canada. In addition, I plan to give an invited talk at the International Parallel and Distributed Processing Symposium (IPDPS) to be held in Florida, USA in May this year.
At the 2023 Memcon conference (March 28-29 in Mountain View, CA), I had the opportunity to see Enfabrica’s Accelerated Computing Fabric Device (ACF) chips. The image below is his Enfabrica display at the Memcon event.
Enfabrica Exhibits at 2023 Memcon
Photo by Tom Coughlin
The company had just come out of stealth mode. The company says its ACF devices are built for use in distributed AI, machine learning (ML), augmented reality, high-performance computing, and in-memory database infrastructure. According to a company release, the company’s ACF device features include:
o Scalable streaming, multi-terabits per second data movement across GPUs, CPUs, accelerators, memory and networking devices.
o Adopt 100% standards-based hardware and software interfaces.
o Eliminate latency hierarchies in today’s top-of-rack network switches, server NICs, PCIe switches, and CPU-controlled DRAM to optimize interface bottlenecks.
o Enable a composable AI fabric of compute, memory, and network resources, from a single system to tens of thousands of nodes.
o Provides contention-free access to 50x+ DRAM expansion over existing GPU networks via Compute Express Link (CXL) bridging.
Enfabrica’s first chip, the Accelerated Compute Fabric Switch (ACF-S), employs standards-based hardware and software interfaces, including multiport 800 Gigabit Ethernet networking and high-radix PCIe Gen5 and CXL 2.0+ interfaces. ACF-S devices provide scalable and configurable high-bandwidth data across any combination of GPUs, CPUs, accelerator ASICs, memory, flash storage, and networking elements participating in AI or accelerated computing workloads. realize the move.
ACF-S enables multi-terabit switching and bridging between heterogeneous computing and memory resources on a single silicon die, providing a more compact solution. Enfabrica’s ACF-S provides headless memory scaling to any accelerator by incorporating CXL memory bridging capabilities, allowing a single GPU rack to directly into local CXL.mem DDR5 DRAM, low latency, contention-free Enables access to a memory capacity that is 50 times greater than before. GPU-native High Bandwidth Memory (HBM).
TrainingCXL announced an AI accelerator device that connects a memory expander to a GPU and processes near-memory training commands. Enfabrica has announced a CXL switch chip that uses standards-based hardware and software for memory pools.
Follow me please twitter or LinkedIn. check out my website.
