Rapid advances in artificial intelligence (AI) numerical forecasting (NWP) have had a major impact on the weather community. Spurred by the release of the Weatherbench framework1 ERA5 Reanalysis Data Set2multiple teams across motivated individuals3university4,5high-tech companies6,7,8nonprofit organizations9and government agencies10 We have developed an AI NWP model that quickly advances the global validation scores of the European Medium-Range Weather Prediction Centre (ECMWF) Integrated Forecast System (IFS) Global Model Headlines to develop an AI NWP model that surpasses the global verification scores11. In addition to improved validation scores, AI NWP models need to have significantly fewer computational resources to run than traditional NWPs7,10. The improved combination of predictive performance at minimal cost opens the door for a surge in new possibilities for how to leverage NWP models with much larger ensembles12faster updates, and potentially improved predictive performance13.
Despite these important advances, a deeper look into published AI NWP models reveals general limitations, particularly with regard to the data used for training. Most of these AI models, including the ECMWF IFS NWP model, rely on some core atmospheric variables, but all other variables are diagnosed from them11. Additionally, the IFS vertical coordinate system uses a hybrid Sigma-pressure level that traces the topography near the surface and relaxes to the pressure level. ERA5 reanalysis used within WeatherBench uses pressure level data instead. This creates a spatially smooth field at all levels, but incorporates extrapolated fields with pressure levels intersecting the topography and with less vertical resolution near the surface.
Existing AI NWP models can accurately predict individual fields. However, these data and modeling choices can cause prediction artifacts that affect the physical consistency of output fields across spatial, vertical levels, and related fields, ensuring that these models are not successfully combined with traditional data assimilation methods.14. Additionally, most publicly available AI NWP models use a 6-hour time step.8each with a mixture of models with different time steps, slowing down the accumulation of regression artifacts. These issues require a more comprehensive investigation of the impact of data, architecture, training procedures, time steps, and post-processing choices on the performance of AI NWP models.
To support the investigation of flaws in the current AI NWP model, we present the Community Research Earth Digital Intelligence Twin (Credit) Framework developed by the Earth Systems (Miles) Group at the NSF National Center for Atmospheric Research (NCAR). The credits act as a basic end-to-end platform designed to be highly customizable throughout the AI NWP modeling pipeline, providing access to both machine learning and earth systems science researchers, extending standard HPC systems and performing inference and analysis on a wide range of computing platforms.
To demonstrate the versatility and functionality of credit platforms, we present two initial AI NWP implementations.5and (2) WXFORMER, a new multi-scale vision trans, has been developed specifically to explore innovative approaches to weather forecasting within the credit framework discussed in detail below. Both implementations demonstrate how credits allow researchers to quickly develop, test, and compare different modeling approaches using standardized datasets and evaluation metrics.
It utilizes a more appropriate training data set that better represents the complexity of atmospheric dynamics. This is complemented by a carefully selected set of input variables that capture important weather processes. Additionally, it employs a computationally efficient and scalable neural network architecture adapted to handle the complexity of weather and climate prediction across a variety of temporal and spatial scales.
In this manuscript, we present a comprehensive assessment of the performance of credit frameworks and WXFormer and Fuxi and Pangue Weather models using Pangue forecasts obtained from WeatherBench2 Dataset.1provides multiple metrics indicating the fidelity of the model across different timescales, from short-term weather forecasts to long-term atmospheric conditions predictions. It also addresses the lack of models and acknowledges the challenges that spread across the wider landscape of automatic regression machine learning atmospheric models. By openly discussing these restrictions, we aim to promote transparent dialogue within our community and pave the way for future improvements in AI-driven atmospheric science.
The credit framework aims to provide a comprehensive research platform for developing and deploying AI-driven models for the Earth System. Here we focus on the atmosphere. It is built on three core components: access to cutting-edge datasets, libraries of advanced models, and infrastructure designed for scalable training.
The credit approach to data management is the basis of its functionality. At NCAR's Derecho HPC, the framework now provides researchers with a critical, high-quality dataset for training accurate atmospheric models, ERA5 and CONUS404.15 More additional additions are planned at the time of this writing. Additionally, it supports the use of user-supported datasets. These datasets are preprocessed and formatted for low latency access through AI training workflows. This feature significantly reduces the entry barrier for new researchers in the field and for researchers who do not have extensive resources for data collection and processing. Advanced users can customize their preprocessing framework within credits. Additionally, Credit's data pipeline is designed to be scalable and consolidated when new datasets become available.
The framework's model library provides a diverse and growing collection of model architectures. This includes simpler, fully convolution-based models such as u-net and its derivatives, as well as state-of-the-art models such as fuxi, The Swin Weather models.16Spherical Fourier Neural Operator (SFNO)17Apply to AI2 Climate Emulator (ACE) models9especially.
Credit provides a scalable training infrastructure that leverages standard high-performance computing (HPC) systems, enabling researchers to take full advantage of the available computational resources to train large-scale AI models. This framework facilitates end-to-end training software and provides templates for users to efficiently train complex models on multiple GPUs without necessarily requiring extensive expertise in parallel computing and HPC environments. By managing much of the complexity of distributed training, credits allow scientists to focus on research goals rather than on calculation details. Additionally, credits support the creation of customized training recipes, increasing flexibility and adaptability in model training.
The framework provides a user-friendly interface and documentation aimed at making it accessible to a wide range of users, from experienced climate scientists to students embarking on their research journeys. Comprehensive documentation, workflow examples, and credit tutorials are available on the GitHub repository and on the related documentation website. These resources are updated regularly to support both new users and advanced practitioners as they deploy models within the credit framework. As we continue to develop and refine these resources, our long-term vision focuses on promoting community-driven credit development. We aim to create an open, collaborative environment where researchers of various levels of expertise can contribute to the evolution of the framework. This approach leverages group expertise to increase credit capabilities and ensures that it addresses the diverse needs of the atmospheric science research community. As our user base grows, we expect community feedback and contributions to play an important role in shaping its functionality, ease of use, and overall orientation.
Beyond its role as a software framework, Credit aims to become an ecosystem that allows researchers, educators and enthusiasts to explore new frontiers of atmospheric, sea and land modeling. It aims to accelerate AI-driven atmospheric science research by providing access to top-notch resources, simplifying technical complexity, and lowering entry barriers. This approach may contribute to advances in understanding weather patterns and climate change, and may play a role in deepening collective knowledge of atmospheric processes on both weather and climate scales.
WXFORMER is the latest advance in deterministic AI-driven atmospheric modeling, specially designed to automatically predict atmospheric conditions with a time resolution of 6 hours or 1 hour (hereinafter referred to as the 6-H or 1-H model). Within the credit framework, models such as WXFormer implement several improvements to mitigate compound error growth, such as spectral normalization of neural network layers, and integrate physical knowledge into the dataset through static variables such as solar radiation above the atmosphere. This model can generate 10-day predictions in 6-hour time steps in 40 seconds on an NVIDIA A100 GPU.
Choosing Crossformer Vision Transformer18 Because WXFormer's backbone also offers several advantages. It facilitates hierarchical attention schemes and reduces patch sizes without dramatically increasing model size or memory footprint. The architecture allows efficient scaling across multiple GPUs, beyond graph-based networks, and enables effective management of data escalation.
The second implementation will adapt to existing Fuxi architectures5 There are several extensions that have been tailored to work best within the credit framework. These changes leverage credit data preprocessing capabilities and calculation infrastructure to improve performance of the original model.
Within the credit framework, both WXFORMER and FUXI incorporate innovative padding techniques to handle the spherical nature of the Earth in global weather simulations. The model uses boundary padding along the map boundary of [0∘–360∘] Longitude and [−90∘to 90∘] Addressing the challenges posed by latitude, polar regions and data lines. To effectively deal with the globular nature of the earth, the credit model uses circular padding along 0∘–360∘ It wraps the data from one end to the other end of the longitude line, simulating periodic boundaries. 180∘ The shift is applied to properly align the data at the poles, the upper row from the Arctic is turned upside down, added above the original data, and the lower row from the Antarctic is similarly added below. When these updates are applied within credits, they provide a stable 10-day forecast. Future publications demonstrate WXFormer's ability to accurately capture a 10-year timescale19.
The implementation of these models demonstrates how credits can quickly develop, test and compare different modeling approaches using standardized datasets and evaluation metrics. By providing this model library, the credits significantly reduce the barriers to entry for researchers interested in AI-driven atmospheric science, while simultaneously providing advanced capabilities to experts in the field.
