Older tools illuminate AI in climate forecast races

The simple truth rattles the world of climate science. Older models may work better than newer models.

Researchers at MIT report that simple physics-based models can outperform large-scale deep learning systems in predicting future regional temperature shifts. The discovery, published in the Advances of Modeling Earth Systems on August 26th, challenges the assumption that artificial intelligence is always a more keen tool. The team also showed that natural shaking in the climate, such as El Niño and random weather noise, can make AI systems look better than they really are. In practice, linear models were often more accurate for temperature, but deep learning had an advantage in rainfall prediction.

Why it's simpler

Climate prediction is the monster in question. Running a full-scale earth system model to see how contamination affects temperatures, it could take weeks on the world's fastest supercomputer. To make this easier to manage, researchers often use “climate emulators” to use peeled models that approximate large simulations. These depend on when policymakers draft emissions targets or climate regulations.

But what if those emulators fail to fire? The MIT team tried to test it. They compared cutting-edge deep learning models with linear pattern scaling (LPS), a statistical method that was decades ago. Using benchmark datasets, LPS has beaten AI in predicting most parameters, especially temperature.

“Large AI methods are very appealing to scientists, but rarely solve whole new problems,” says Björn Lütjens, lead author of IBM Research.

At first glance, the results were inexplicable. The precipitation is messy and nonlinear, so deep learning should be excellent. However, researchers have found that natural fluctuations (the chaotic ups and downs of the climate) drive AI to overfitting. Essentially, instead of capturing the signal, I remembered noise.

Building a more fairer test

To fix this, the team built a new rating system using more climate simulations. When they did so, the deep learning model improved and edged the LPS with precipitation prediction. Still, LPS remained more accurate to surface temperature.

lesson? The benchmarking method is important. “It's important to use modeling tools that are suitable for the problem, but to do so, you need to set the problem in the right way first,” says Professor Noelle Selin, co-author and director of the Center for Sustainability Science Strategy.

“We are trying to develop a model that is useful and relevant for decision makers,” Serin said.

Policy and Science Impact

This research is not a rejection of AI, but a warning tale. Climate science already has strong physical laws. The challenge is to combine these laws with machine learning and machine learning to avoid overfitting and misuse.

The key findings are as follows:

• The linear regression model outperformed deep learning in three of the four major climate variables, including temperature.
• Deep learning struggled with internal fluctuations and often overfitted climate noises like the El Niño/La Niña cycle.
• With more data, deep learning gained the advantage of predicting local precipitation, but not at temperature.

For policymakers, this means caution is needed before trusting the largest AI models to promote climate planning. A simpler emulator can draw a true picture of how emissions change temperatures in a region. At the same time, deep learning remains promising on troublesome questions such as extreme rainfall and aerosol impact.

remove

This MIT study shows that simpler physics-based models can outperform large-scale AI systems in predicting local temperature changes, but deep learning is more effective than rainfall. The results highlight the importance of better benchmarks to guide the modeling approach used.

The next step is to develop an improved climate emulation tool that combines the strengths of both approaches.

Journal: Journal of Advanced Modeling Earth System
doi:10.1029/2024MS004619