Like many HPC and AI system builders, we can’t wait to see what AMD’s ‘Antares’ Instinct MI300A hybrid CPU-GPU system-on-chip looks like in terms of performance and price.
Also, with the ISC 2023 supercomputing conference just a few weeks away, Bronis de Supinski, Chief Technology Officer at Lawrence Livermore National Laboratory, spoke at the event about the future “El Capitan” exascale system, which will be its flagship machine. I’m here. As for his MI300A variant of Antares GPU, it’s on our minds.
So, just for fun, I pulled out my trusty Excel spreadsheet and tried to estimate the feed and speed of the heart of the El Capitan system, the MI300A GPU. Yes given that AMD will likely talk a bit more about the MI300 series GPUs in his ISC 2023 and beyond and eventually we’ll know exactly how this compute engine is designed And this is probably stupid. But can the MI300 series compete with Nvidia’s ‘Hopper’ H100 GPU accelerators and, perhaps more importantly, with Hopper’s tightly-anchored 72-core ‘Grace’ Arm CPU combination? Please, many people have asked questions. Create a Grace-Hopper hybrid CPU-GPU complex combining H100 GPUs. This works on par with his MI300A, which will be deployed on El Capitan, and other hybrid CPU-GPU machines running HPC and AI workloads. Alongside.
There is also a strong push towards GPU computing, driven by the explosive growth of AI training for generative AI applications based on large language models and AMD’s desire to play a greater role in AI training using GPUs. Considering the demand, we think the demand will surpass Nvidia. This means that AMD’s GPUs will get some AI supplies, even though the Nvidia AI software stack has a big edge over AMD. The predecessor ‘Aldebaran’ GPUs have already seen impressive success in AMD’s HPC designs, especially in Oak Ridge National Lab’s ‘Frontier’ exascale systems. A more loosely coupled hybrid computing engine. (There are others.) And we think the strong demand for his Nvidia GPUs for AI workloads really leaves room for AMD to score some deals as demand outstrips supply. .
People today have less patience to add generative AI to their workloads than they did in the late 1990s and early 2000s, adding web infrastructure to modernize their applications and deploying interfaces on the internet. I don’t think so. The difference this time is that the data center itself does not turn into a general-purpose X86 compute board, but rather an ecology of competing and complementary architectures woven together for the best overall cost-effectiveness. It’s becoming a system. Accommodate a variety of workloads.
Not much is known about the MI300 series yet, but back in January AMD made some small talk about the device. We have an image of the device and it is known to have 8x the AI performance per watt and 5x the AI performance of his existing MI250X GPU accelerator used in the Frontier system. The entire MI300A complex is known to have 146 billion transistors spread across 6 GPUs and 2 CPU chiplets. Most of that transistor count is believed to be implemented in the four 6-nanometer tiles that interconnect the CPU and GPU compute elements where the Infinity cache is also implemented. It’s hard to say how many transistors this cache will consume, but we’re looking forward to finding out.
By the way, MI300A seems to refer to the APU version of AMD’s flagship parallel computing engine, that is, the combination of CPU and GPU cores in one package. This means there will be a non-APU, GPU-only version of the Antares GPU. There are probably up to 8 GPU chiplets on top of these 4 interconnects and cache chips as shown below.
To be precise in AMD’s words earlier this year, its 8X and 5X numbers were based on testing the MI250X GPU and modeled performance of the GPU portion of the MI300A complex. And to be very specific, AMD says: 306.4 is his estimated TFLOPS provided based on 80% of peak theoretical floating point performance). MI300 performance is based on preliminary estimates and expectations. Final performance may vary. “
So, here’s a table that estimates what the MI300A’s feed and speed might look like, considering what AMD has said so far.
Where MI250X’s FP16 performance is 383 teraflops, that’s 8X multiples, including a downshift to FP8 data format and processing, and MI300A with sparse support (offering another 2X of that 8X multiple) peaks at 3,064 teraflops means that you can push If the MI250X is rated at 560 watts, then the GPU portion of the MI300A should be running at 900 watts for the 5x performance-per-watt improvement AMD is talking about.
If all the above are true, the MI300A CPU complex should have 4x the performance of the MI250X. Also, assuming the clock speed remains the same at 1.7 GHz, MI300A’s six GPU chiplets would require four times more compute units and streaming processors than MI250X. If AMD can increase the clock speeds, I think it’s unlikely a shift from 6-nanometer to 5-nanometer process, but it’s not a huge jump.
Just as Nvidia doubled the performance of its H100 GPU’s matrix math unit compared to its vector unit, we believe AMD will do the same with its MI300A hybrid compute engine. If in matrix units he sees a 4x improvement, in vector units he may only see a 2x improvement. This is another way of saying that many of his HPC workloads won’t get as fast as AI training workloads unless they are fine-tuned to run on the Matrix Unit.
Now let’s talk money.
When the MI250X first shipped to Oak Ridge in December 2021 and the Frontier machine was built, our analysis estimated the list price for one of these GPU motors could be in the $14,500 range. Nvidia “Ampere” A100 SXM4 GPU accelerator. It sold for $12,000 at the time. Following his H100 announcement in March 2022, the price of his top-of-the-line H100 SXM5 (which cannot be purchased separately from the HGX system board) could go from $19,000 to $30,000. It is estimated that there is The PCI-Express version of the H100 GPU is probably worth between $15,000 and $24,000. At the time, high demand had pushed the price of the A100 SXM4 up to around $15,000. And just a few weeks ago, a PCI-Express version of the H100 was auctioned on eBay for over $40,000 each. it’s crazy.
It’s worse than the used car market here in the US, a kind of inflation from too much demand and too little supply. A situation that vendors prefer when they know they can’t make enough units anyway. Hyperscalers and cloud builders allocate access to GPUs among their developers. It wouldn’t surprise me to see the price of GPU capacity rise in the cloud.
In terms of FP8 performance with sparsity turned on, the MI300A delivers around 3 petaflops at theoretical peak performance, but probably around 5.4 TB/s bandwidth for 128 GB of HBM3 memory To do. The Nvidia H100 SXM5 unit has 80 GB of HBM3 memory with 3 TB/s bandwidth and is rated at 4 petaflops peak performance with sparsity turned on at FP8 data resolution and processing . AMD devices have a 25% drop in peak performance, but a 60% increase in memory capacity, and perhaps an 80% increase in memory bandwidth if all eight HBM3 stacks on the device can be fully implemented. (I hope so.) Many AI shops are willing to sacrifice a little bit of peak performance for more memory bandwidth and capacity that can help improve performance in real-world AI training. I’m here.
What we can say for sure is that El Capitan is the first line of MI300A compute engines, and 22,000 sockets are needed to break the 2.1 exaflops peak in plain vanilla 64-bit double precision floating point, and in this case , the socket is a node. The current “Sierra” system is nearly five years old and has 4,320 nodes, as El Capitan came to market about a year later than planned (but probably still within the $600 million budget). , each with two and four IBM Power9 processors. Nvidia’s “Volta” V100 GPU accelerator. That’s a total of 17,280 Sierra GPUs, and if our guesses about the MI300A’s FP64 performance are correct (and this is the first time we’ve admitted that this is just a hunch), it’s likely that El Capitan’s GPU socket will do just that. 27% more than Sierra. However, each El Capitan socket has his 6 logical GPUs, so that equates to 132,000 GPUs to deliver 2.1 exaflops. This delivers 16.9x raw he FP64 performance for 4.8x the price across the two systems, delivered by a 7.6x increase in GPU concurrency. El Capitan should perform at least 10x better than Sierra in a thermal envelope of less than 40 megawatts. If all this is true, 2.1 exaflops of El Capitan would be about 25 megawatts for the compute engine alone.
And for this whole price check, if 85% of the cost of an El Capitan machine is the CPU-GPU compute engine, and there are 22,000 of them, that’s about $23,200. And no hyperscaler or cloud builder is paying less than the US national labs that basically back AMD’s forays into the upper tier of HPC. (That’s a lot of “what ifs,” and we know it all too well.)
In the past, we actually calculated GPU list prices from supercomputing deals by revoking HPC National Labs deep discounts. For example, in the Volta V100 accelerator used in Sierra, the GPU was listed for around $7,500 but was sold to Lawrence Livermore and Oak Ridge for around $4,000. As such, the MI300A’s list price could exceed $40,000 if older discount levels prevail. With AMD adding more compute to his MI300A engine and the price per unit was also significantly lower, we believe the discount isn’t too steep.
Recall that when the first El Capitan deal was announced in August 2019, with delivery in late 2022 and approval by the end of 2023, it was designated as a 1.5 exaflops machine. please give me. Persistence It consumes about 30 megawatts of power just to run the system.
This leaves two questions. 1: How many MI300A devices can AMD make, and if it’s a lot more than what’s slated for El Capitan, he can set a price and sell them all. And 2: Will AMD sell them at an aggressive price, or will they impose a price the market can bear?
The second question is not difficult to answer, is it? Not in this bullish GPU market, where AI is completely recession-proof. AI could even accelerate the recession if it becomes increasingly successful at replacing humans. . . so far there has been no real or AI-accelerated recession.
