Microsoft uses AMD’s AI-optimized Helios racks on Azure

Microsoft Corp. plans to utilize Advanced Micro Devices Inc.’s upcoming Helios rack design for some Azure services.

The cloud and operating system giant today announced the partnership along with three new instance families.

Helios is a reference design, a blueprint that AMD’s manufacturing partners can use to build data center racks. Each system includes 72 of the chipmaker’s upcoming Instinct MI455 graphics processing units. AMD has so far released only a few details about its GPUs. It features 432 gigabytes of HBM4 memory, 19.6 terabits per second of bandwidth, and a new core architecture called CDNA 5.

Helios’ GPU is supported by a Pensando data processing unit and an Epyc central processing unit.

The Pensando chip is optimized for infrastructure management tasks such as coordinating storage equipment and encrypting network traffic. AMD says it can run these workloads more efficiently than the CPU. This reduces costs and makes more CPU capacity available to customer applications.

Helios’ CPUs are from AMD’s upcoming Venice data center processor series. In May, the company began ramping up production of chips using Taiwan Semiconductor Manufacturing’s 2-nanometer process. The Venice series also uses a second TSMC technology called SoIC, which allows chiplets to be stacked on top of each other.

Helios organizes CPUs, GPUs, and DPUs into modules called trays. The tray is wider than a standard rack server and can accommodate more hardware. It uses liquid cooling to dissipate heat from the chip and exchanges data using an open-source networking protocol called UALoE.

“AMD and Microsoft have been building high-performance infrastructure together for years, and today we are extending that partnership across the full stack of AMD AI solutions on Azure,” said Lisa Su, CEO of AMD.

AMD plans to start shipping Helios racks to Microsoft and other customers later this year. The tech giant plans to use these systems to power a new Azure instance family called the ND MI455X v7 series. According to Microsoft, virtual machines are optimized for inference workloads such as artificial intelligence agents and search tools.

The company debuted the ND MI455X v7 series along with two other instance families that also run on AMD silicon.

The HDv2 series is optimized for tasks that AI applications perform on the CPU rather than on the CPU. This includes the process of preparing datasets for analysis by AI agents. Each instance includes up to 500 Epyc Vulcan cores, 4 terabytes of memory, and 32 terabytes of flash storage.

The third addition to Azure’s virtual machine portfolio is an instance series called HXv2. This is an improved version of the existing Azure instance series that is optimized for EDA (electronic design automation) applications. These are programs that engineers use to design chips. Microsoft says HXv2 supports a wide range of workloads, including scientific simulation.

Each HXv2 virtual machine has 176 Epyc Vulcan cores with clock speeds over 5 GHz. According to Microsoft, each core will have 50% more cache than previous generation hardware. Customers can configure virtual machines with up to 4 GB of memory.

“Significant increases in per-VM and per-core performance and the inclusion of 800 Gb InfiniBand enable large-scale MPI-based simulations, making HXv2 ideal for a wide variety of HPC customers,” Scott Guthrie, Microsoft’s executive vice president of cloud and AI, said in a blog post.

The company’s new collaboration with AMD also extends to a technology called Azure Boost. Offloads the computations associated with running virtualization software from the CPU to a more efficient dedicated chip. Microsoft plans to work with AMD to optimize Azure Boost for AMD’s products.

photograph: AMD