The digital world cannot exist without the natural resources to run it. How much does the technology you're using to build and run AI cost?
Machine learning has core concepts that I often talk about to the public to clarify the philosophy behind what we do. The idea is that the world changes around every machine learning model. because Therefore, the world that a model is trying to emulate and predict is always the past, not the present or the future. In some ways models are predicting the future (as we often think), but in many other ways they are actually trying to take us back in time.
I want to talk about this because philosophy around machine learning helps give us a real perspective, not just as machine learning practitioners, but as users and subjects of machine learning. Regular readers will know that I often say “machine learning is us.” That is, we generate data, train, and consume and apply the model output. The model tries to follow our instructions using the raw materials we provide, but we have almost complete control over how that happens and what the outcome is.
Another aspect of this concept that I find useful is that it reminds us that models are not isolated in the digital world, but are in fact deeply intertwined with the analog physical world. That's the point. After all, if your model has no impact on the world around us, the question arises why it exists in the first place. Ultimately, the digital world is only separated from the physical world in a limited, artificial sense of how users/developers interact with it.
This last point is what I want to talk about today. How is the physical world formed and how does it inform machine learning? And how will ML/AI impact the physical world? In my last article, I promised to talk about how the resource limitations of the physical world intersect with machine learning and AI. Proceed there.
This is probably obvious after a little thought. There's a joke going around that you can defeat your sentient robot overlords by simply turning off your computer or unplugging it. But all jokes aside, there is a kernel of truth in this. Those of us involved in machine learning, AI, and computing in general are completely reliant on natural resources like mined metals and electricity to keep our industries afloat. This has some similarities with an article I wrote last year about how human labor is necessary for machine learning to exist, but today I'm going in a different direction and explaining how we need human labor to exist. We will discuss two important areas that we should recognize as essential to our society. Work – mining/manufacturing and energy, mainly in the form of electricity.
There is a wealth of research and journalism on both of these areas if you look for it. This not only relates directly to AI, but also to earlier technology booms such as cryptocurrencies. Cryptocurrency has a lot in common with AI. usage of that resource. We will provide an overview of each area and include quotes for further reading so you can explore the details and get to the source of the scholarship. However, it is difficult to find studies that take into account the AI boom over the past 18 months, so we expect that some of this research underestimates the impact of new technologies in the field of generative AI.
What does it take to manufacture GPU chips? We know that these chips are helping develop the latest machine learning models, and Nvidia, the largest producer of these chips today, is taking advantage of the cryptocurrency boom. It has ridden the AI boom to become one of the most valuable companies in existence. Their stock price has risen from $130 per share in early 2021 to $877.35 per share as I write this in April 2024, with a reported market capitalization of over $2 trillion. I am. In the third quarter of 2023, he sold over 500,000 chips for over $10 billion. His total H100 sales in 2023 are estimated at 1.5 million units, and he is easily expected to surpass that number in 2024.
GPU chips contain a variety of specialty raw materials that are somewhat rare and difficult to obtain, such as tungsten, palladium, cobalt, and tantalum. Other elements, such as mercury and lead, may be more readily available but pose significant health and safety risks. Mining these elements and compounds has significant environmental impacts, including emissions and environmental damage to the areas where they are mined. Even the best mining operations change the ecosystem significantly. This is compounded by the risk of so-called “conflict minerals,” or minerals mined under conditions of human exploitation, child labor, slavery, etc. (Credit: Nvidia has been very vocal about avoiding the use of such minerals, specifically naming the Democratic Republic of Congo.)
Moreover, after the raw materials are mined, all these materials must be processed very carefully in order to produce small and extremely powerful chips that perform complex calculations. Over the past 150 years of industrial history, workers have to take significant health risks when working with heavy metals such as lead and mercury. Nvidia's chips are primarily manufactured in factories in Taiwan run by a company called Taiwan Semiconductor Manufacturing Company (TSMC). Nvidia is able to avoid criticism about manufacturing conditions and emissions because it doesn't actually own or operate the factories, and data is difficult to obtain. The electricity required to make this is also not on Nvidia's books. Side note: TSMC has reached maximum capacity and is working on increasing capacity. In parallel, NVIDIA plans to begin cooperation with Intel on manufacturing capacity next year.
Once a chip is manufactured, it has a very long service life of 3 to 5 years if properly maintained. However, Nvidia is constantly producing new chips that are more powerful and efficient (2 million a year is a lot!). Therefore, the lifespan of the tip can be limited by aging and wear. When the chip is no longer useful, it goes into the so-called “e-waste” pipeline. Theoretically, many of the rare metals in chips should have some recycling value, but as you can imagine, chip recycling is a highly specialized and difficult technical challenge, and is a major contributor to overall e-waste. Only about 20% is recycled. Complex things like phones and other hardware. The recycling process also requires workers to disassemble equipment, which brings them back into contact with heavy metals and other elements that were involved in its production in the first place.
On the other hand, if the chips are not recycled, they can end up in landfills or be incinerated, potentially leaching heavy metals into the environment via water, air, or both. This often occurs in developing countries and directly impacts the areas where people live.
However, most of the research on machine learning's carbon footprint and its general environmental impact is related to power consumption. So let's look in that direction.
Power consumption is definitely an issue in AI-equipped rooms, given the necessary hardware to do the job. Training large language models consumes enormous amounts of power, but serving and deploying LLM and other advanced machine learning models also creates a power sinkhole.
For training, one research paper states that training GPT-3 with 175 billion parameters requires approximately 1,300 megawatt-hours (MWh) or 1,300,000 KWh of power to operate. Compare this to his GPT-4 which uses 1.76. Trillion parameters, and the estimated power consumption for training was between 51,772,500 and 62,318,750 KWh of power. For comparison, the average American home uses just over 10,000 KWh of electricity per year. From a conservative perspective, training GPT-4 once could allow him to power approximately 5,000 American homes for a year. (This does not take into account all the power consumed by the preliminary analysis and testing that will almost certainly be required to prepare the data and prepare it for training.)
Given that the power usage between training GPT-3 and GPT-4 has increased approximately 40 times, the future power consumption involved in the next versions of these models and the training models that generate videos and images We should also be concerned about the consumption of. , or audio content.
The power consumption of inference tasks increases rapidly after the training process, which only needs to be run once in the model's lifetime. That is, the cost you incur every time you ask Chat-GPT a question or try to generate an interesting image. AI tools. This power is absorbed into the data center where the model is running, so the model can deliver results worldwide. The International Energy Agency predicts that by 2026, data centers alone will consume 1,000 terawatts, roughly equivalent to Japan's electricity usage.
Leading companies in the AI industry are clearly aware of the fact that this kind of increase in power consumption is unsustainable. It is estimated that data centers consume 0.5% to 2% of the world's electricity usage and could consume 25% of the US electricity usage by 2030.
The state of the electricity infrastructure in the United States is not very good. Of course, we are trying to add more renewable electricity to the grid, but it's no wonder we're not known as a country that manages our public infrastructure well. Texas residents in particular are aware of the fragility of their power systems, but climate change is occurring across the United States in the form of more extreme weather events and more power outages.
It remains to be seen whether investments in power infrastructure can match the surge in demand brought on by AI tools, but government action is needed to get there, so pessimism is justified. Of course.
In the meantime, even if we can produce electricity at the rate we need it, these AI tools can help reduce global carbon emissions until renewable, emission-free power sources become scalable. It will increase significantly. Carbon emissions per KWh of electricity are estimated at 0.86 pounds, and training GPT-4 releases more than 20,000 tons of carbon into the atmosphere. (In contrast, the average American emits 13 tons per year.)
As you can imagine, I'm not here to argue that we should abandon machine learning because it consumes natural resources. I believe that the workers who make our lives possible should be given significant workplace safety measures and compensation commensurate with the risks they take. I also believe that renewable power sources should be a high priority in combating preventable human-induced climate change.
But the reason I talk about things like this is that knowing how dependent our work is on the physical world, on our natural resources, on the earth, makes us humble and grateful for what we have. Because you should be able to be grateful. When you perform training or inference, or use Chat-GPT or Dall-E, you are not the endpoint of the process. Your actions have downstream effects. It's important to be aware of that and make informed decisions accordingly. You might end up borrowing someone else's GPU for a few seconds or hours, but it still uses power, wears out the GPU, and eventually needs to be retired. Part of being an ethical global citizen is thinking about your choices and considering the impact on others.
Additionally, if you want to learn more about the carbon footprint of your own modeling efforts, we have a tool for that: https://www.green-algorithms.org/
