by Chris Noble
Since its rapid penetration into public consciousness last fall, generative artificial intelligence (AI) has fueled many exciting advances and innovations, as well as employment opportunities across various industries. has fostered serious dialogue about its impact on However, the impact of generative AI on the environment and how companies can use AI responsibly and sustainably is less discussed. The information technology industry is already estimated to account for 2-4% of total global greenhouse gas emissions, more than the aviation industry.[1] – And computing in 2040 will require more energy than is being produced today.[2]
AI consumes energy in two main ways: training and inference. Training is the process by which the AI learns how to identify patterns and relationships between data points. The more parameters a model uses, the more likely it is to give an accurate answer during the inference stage. However, training with larger datasets and more parameters increases the required computational power exponentially.
The inference stage refers to using a post-trained AI model to make predictions or generate content based on prompts. Since the system has already set parameters and learned patterns, each inference requires significantly less computational power. However, responding to simple prompts usually requires some reasoning. Doing this quickly to achieve the desired result can consume a large amount of computing resources, especially if the system is serving many users simultaneously.
When it comes to training generative AI models, the process consumes much more computing power and energy than predictive AI technologies. GPT-3, on which ChatGPT is partly based, used an estimated 552 tCO, he reported by Google and University of California, Berkeley researchers.2e of CO2 Energy consumption during training corresponds to 1,287 MWh.[3] This is equivalent to the electricity consumed by 121 US households in one year.[4]!
Similarly, Meta’s OPT-175B was developed for an estimated 75 tCO2However, including ablation, baseline and downtime, this doubles to about 150 tCO2e.[5] Meta researchers report that the company’s AI training saw a 3.2x increase in data ingestion bandwidth demand from 2019 to 2021 and a 2.9x increase in training infrastructure capacity over 1.5 years.[6] Equally alarming is a 2018 analysis by ChatGPT developer OpenAI. This analysis showed that since 2012, the amount of computing used in large-scale AI training runs has grown exponentially with his doubling time of 3.4 months. For comparison, Moore’s Law predicted that computational efficiency he would double every two years.[7]
Unfortunately, there is even less data from which we can make inferences about the energy consumption and environmental impact of generative AI. A recent study by Northeastern University and the Massachusetts Institute of Technology showed that reasoning has a significantly greater impact on energy expenditure than training.[8] AWS and Nvidia estimate that inference can be up to 80-90% of the total operating cost of deep learning.[9][10]
At Google, machine learning (ML) energy use across research, development, and production will account for 10-15% of Google’s total energy use, according to a weeklong study conducted in April 2019-2021. % was. About 3/5 of Google’s ML energy usage was used for inference and 2/5 for training.[11] Similarly, Meta found that inference accounts for 50-65% of the operational carbon footprint of machine learning, and growing inference demand will lead to a 2.5x increase in inference infrastructure capacity from 2019 to 2021. Increased.[12]
Exact numbers remain elusive, but it is nonetheless clear that this generative AI boom will only increase carbon emissions in the IT sector. And this doesn’t even consider the impact on water. Researchers at the University of California, Riverside and Arlington found that GPT-3 training could directly consume 700,000 liters of clean fresh water, while ChatGPT inference could consume 500 mL of bottled water in a short 20-minute conversation. We estimate that it can be consumed. -50 questions and answers.[13] So what should companies do if they want to take advantage of generative AI without turning back on their sustainability efforts?
Overall, companies should use AI judiciously and develop it to perform in the most efficient way possible.
- Develop a model that fits your use case – It is important to tune your model and dataset to match your use case goals and avoid overtraining your system with irrelevant data and parameters. Serving the entire internet to a generative AI model consumes an enormous amount of computing power. there is no need. Similarly, predictive AI models work fine to optimize data center power usage without training on data that tracks seismic activity in the earth’s crust.
- Evaluate parameters need – Evaluate the trade-off between accuracy and efficiency when deciding how many parameters to include. Do we really need to use 175 billion parameters, or can we achieve nearly the same level of accuracy with fewer parameters?
- Carefully retrain your model – Determine the frequency of retraining required and schedule training during off-peak hours when regions powered by renewable energy and/or less fossil fuels are required to meet fluctuating demand. Think critically about when, where, and how often you retrain your model.
- Run inference in areas with clean energy – Inference requires more processing power than traditional queries when processing data, so enterprises should consider directing inference traffic to locations that run on clean energy. Individual inferences are not as computationally intensive as training, but can grow cumulatively as a generative AI model continues to run.
- Don’t be fooled by the AI hype – Understand the problem and design with a specific use case in mind. Avoid continuous data ingestion for simple problems that do not require continuous data ingestion.
Generative AI is already revolutionizing the way humans work, but as we move toward a more sustainable future, we need to keep in mind the impact of generative AI on the climate.
[1]https://www.sciencedaily.com/releases/2021/09/210910121715.htm
[2] Report updated September 2020 (see pages 17-18 of summary report: https://www.src.org/about/decadal-plan/
[3]https://arxiv.org/ftp/arxiv/papers/2104/2104.10350.pdf
[4]https://www.eia.gov/tools/faqs/faq.php?id=97&t=3#:~:text=In%202021%2C%20the%20average%20annual,about%20886%20kWh%20per%20month
[5]https://arxiv.org/pdf/2205.01068.pdf
[6]https://research.facebook.com/publications/sustainable-ai-environmental-implications-challenges-and-opportunities/
[7]https://openai.com/research/ai-and-compute
[8]https://semiengineering.com/ai-power-consumption-exploding/
[9]https://aws.amazon.com/machine-learning/elastic-inference/
[10]https://www.forbes.com/sites/moorinsights/2019/05/09/google-cloud-doubles-down-on-nvidia-gpus-for-inference/?sh=56c3e48f6792
[11]https://www.techrxiv.org/articles/preprint/The_Carbon_Footprint_of_Machine_Learning_Training_Will_Plateau_Then_Shrink/19139645;
[12]https://research.facebook.com/publications/sustainable-ai-environmental-implications-challenges-and-opportunities/
[13]https://themarkup.org/hello-world/2023/04/15/the-secret-water-footprint-of-ai-technology
