Amazon (AMZN 1.96%) In his second annual letter to shareholders, CEO Andy Jassy offered a number of useful tips for investors. One of which he did was with the modern semiconductor industry, the backbone of all computing his technology. Amazon has actually invested in its own semiconductor designs for years, increasingly into high-performance machine learning processors used to power Large Language Models (LLM), services such as ChatGPT. Emphasis. It is based on.
NVIDIA (NVDA 0.95%) The company’s graphics processing units (GPUs) have taken the lead in this sector, and the stock has recently boomed on optimism about long-term earnings growth prospects from LLM. But will Amazon’s investment in chips be a problem for Nvidia?
Jassy on what makes a good investment
Amazon Web Services (AWS) was a pioneer in the rapidly expanding cloud industry. Today it is his AWS that pays the bills in the Amazon empire and helps fund many other business investments.
But how did Amazon decide to invest in new ventures like semiconductor design? There are four key metrics, as Jassy explained in her 2022 annual shareholder letter.
• If successful, will it scale and provide a reasonable return on invested capital?
• Are enough opportunities available today?
• Do you have a differentiated approach?
• And do we have competence in that area? If not, can we get it quickly?
Apparently, when AWS considered designing its data center chips, the computing hardware that powers the cloud, the answer to each of these four questions was yes. To make his silicon dreams come true, he quietly acquired Annapurna Labs, an Israeli chip design startup, for $350 million in 2015.
How Amazon’s investment paid off
Annapurna Labs has designed numerous chips for AWS. This includes Graviton processors.This is an ARM based chip that replaces the CPU provided by AWS intel and AMD. But what about computing accelerators like Nvidia’s GPUs powering new AI services like ChatGPT?
That’s where AWS Trainium and Inferentia chips come in. Neither of these computing accelerators can match his Nvidia’s latest and greatest design in pure computing power (neither is). alphabetGoogle Cloud internal chip). But that wasn’t Amazon’s main goal when developing Trainium and Inferentia. The goal was cost performance.
As its name suggests, Trainium aims to train how LLMs behave using massive amounts of data. Inferentia is for inference and is where most of the computing work is done after AI model training. Inference is how a trained AI program makes decisions based on what it has already learned (such as asking ChatGPT a question and getting an answer back).
AWS itself uses Trainium and Inferentia, but has made more cost-effective accelerators available to customers. In a letter to shareholders, Jassy said generic AI models trained on Trainium are “up to 140% faster” and “up to 70% cheaper” than similar his GPU systems. I’m here. And when it comes to AI inference, since its introduction in 2019, the company’s Inferentia chip “has saved him more than $100 million in capital expenditures for companies like Amazon,” Jassy said.
Simply put, Amazon’s $350 million investment in Annapurna in 2015 looks like an incredible long-term payoff for AWS and shareholders.
Press Nvidia to improve?
Competition is great because it keeps business leaders pushing their companies toward continuous improvement. Nvidia is trying to make hay out of its cutting-edge GPUs for advanced AI, but there are plenty of other chips that can continue to improve. For example, in March we launched a new L4 GPU for AI inference. It has a software stack to optimize a variety of AI workloads and lower total cost of ownership for cloud providers and customers.
In fact, while Amazon AWS and other cloud providers are making waves with announcements about their own silicon designs, AWS continues to be Nvidia’s primary customer. AWS’ in-house chips now fill a small niche in the cloud giant’s operations.
Of course, increased competition from fellow tech giants is a big risk for Nvidia. But far from defenseless. Additionally, cloud computing and AI are still in the early stages of the adoption curve. As Jassy pointed out at the end of his letter to shareholders, his AWS revenue in 2022 was his $80 billion, but “about 90% of his IT spending globally” still moved to the cloud. not done in an on-premises system.
In other words, Amazon AWS has made rapid progress in designing chips in-house, but there is a lot of new business. Nvidia will be fine.
Alphabet executive Suzanne Frey is a member of The Motley Fool’s board of directors. John Mackey, former CEO of Whole Foods Market, an Amazon subsidiary, is a member of The Motley Fool’s board of directors. Nicholas Rossolillo and his clients have held positions at Advanced Micro Devices, Alphabet, Amazon.com, and Nvidia. The Motley Fool has positions in and recommends Advanced Micro Devices, Alphabet, Amazon.com, and Nvidia. The Motley Fool recommends Intel and recommends the following options: Intel’s Jan 2023 long call of $57.50 and Intel’s Jan 2025 long call of $45. The Motley Fool’s U.S. headquarters has a disclosure policy.
