[Research Article] Quantifying Uncertainty in Geospatial AI/ML Applications: Methods, Metrics, and Open Source Support with Air Quality Use Case

New research published in big earth data We systematically evaluate common uncertainty quantification (UQ) methods and metrics for AI/ML-based geospatial applications to address the challenges of forecast uncertainty, reliability, and real-time integration in complex Earth system modeling. Using a PM2.5 calibration case study, we demonstrate that deep ensembles and Bayesian neural networks implemented in TensorFlow achieved the best reliability and calibration performance, while also highlighting the framework-specific differences between TensorFlow and PyTorch for geospatial UQ applications.

quotation

Malarvisi, A. S., Smith, K., and Yang, C. (2026). Quantifying uncertainty in geospatial AI/ML applications: Open source support with methods, metrics, and air quality use cases. big earth data1-34. https://doi.org/10.1080/20964471.2026.2629680

abstract

The deployment of AI/ML models in geospatial applications faces challenges related to trustworthiness and trustworthiness due to model processing and dataset uncertainties. These uncertainties are further exacerbated by the complexity of Earth’s systems and constraints such as data quality and measurement accuracy. Despite the growing interest, the field lacks a systematic comparison of uncertainty quantification (UQ) methods tailored to the unique challenges of geospatial modeling, especially in real-time settings. These challenges need to be addressed by integrating UQ into geospatial workflows to help stakeholders and users understand model uncertainties and make decisions with confidence. This paper addresses these gaps by systematically evaluating three popular UQ techniques, focusing on their suitability for geospatial applications and their ability to deal with prediction uncertainty. Furthermore, we analyzed various UQ metrics and highlighted the role of UQ metrics in evaluating predictive ability and calibration. However, improvements are needed to establish an integration framework and streamline the integration of real-time and large-scale applications. A case study on PM2.5 calibration demonstrated the practical utility of UQ in increasing the reliability of air quality (AQ) data. The results highlight that TensorFlow’s Deep Ensembles (DE) achieved the strongest performance, followed by TensorFlow’s BNN, which provided reliable calibration and accuracy for the case study. Next came TensorFlow’s MCD, which achieved stable performance but with lower adaptive uncertainty at extreme PM2.5 values. PyTorch’s MCD had the worst performance, poor accuracy, and unreliable calibration. These results show that TensorFlow consistently outperforms PyTorch across all methods, highlighting framework-specific differences in the implementation of UQ. Overall, this study conducted a comprehensive review and experimental study of UQ methods and frameworks, using PM2.5 calibration as an example, and provided important insights into its theoretical foundation, practical implementation, and performance in geospatial applications.
#Earth science #Remote sensing #Earth observation #GIS #Data analysis #Big data #Visualization

big earth data is an interdisciplinary open access journal that aims to provide an efficient and high-quality platform to facilitate the sharing, processing and analysis of Earth-related big data, thereby revolutionizing the perception of the Earth system. The journal publishes a wide range of content, including research papers, review articles, data notes, technical notes, and perspectives. Currently included in ESCI (IF=3.8, Q1), Scopus (CiteScore=9.0, Q1), Ei Compendex, GEOBASE, and Inspec. From 2023, big earth data has announced a new award series for authors: Best Paper Award and Outstanding Paper Award.