AI model uses physics to automatically correct remote sensing data

Atmospheric turbulence, temperature changes, and gases such as water vapor, carbon dioxide, ozone, and methane absorb, reflect, and scatter sunlight as it passes through the atmosphere, reflects off the Earth's surface, and is collected by sensors on remote sensing satellites. As a result, the spectral data received by the sensors is distorted.

Scientists know this, and have devised several ways to account for the adverse effects that the atmosphere has on remote sensing data.

“This problem is as old as aerial photography,” says James Koch, a data scientist at the Pacific Northwest National Laboratory (PNNL), who has developed a new way to address the problem, using a branch of artificial intelligence called physics-based machine learning, which also enhances remote-sensing capabilities.

Koch presented a paper describing his physics-based machine learning framework at the International Geoscience and Remote Sensing Symposium, held July 7-12 in Athens, Greece. The research is part of PNNL's Remote Exploitation capability and was supported by the National Security Agency's Laboratory Directed Research and Development portfolio.

Scientists can solve the problem of air pollution because they understand the physics of how the atmosphere distorts sunlight as it passes through it, which allows them to remove the atmospheric effects from the data collected by the sensor, a process called atmospheric correction.

To perform atmospheric correction, prior knowledge of the atmospheric transmission profile is usually required, which describes the properties and composition of the atmosphere at different altitudes and indicates how different wavelengths of light interact with the atmosphere.

The process of creating atmospheric transmission profiles without prior knowledge is an area where Koch’s AI technology could be a game-changer.

Currently, many atmospheric correction applications rely on off-the-shelf tools that use generic, statistically-based atmospheric profiles. These tools are sufficient for time-sensitive tasks such as disaster response monitoring and are cost-effective when mapping large areas. Applications where high accuracy is paramount, such as target detection, require the creation of high-fidelity profiles, which are data-intensive and computationally expensive.

Physics-based machine learning

To train and evaluate his machine learning pipeline, Koch used a dataset of labeled aerial images of Cooke City, Montana, that contain cars and pieces of fabric with known spectral characteristics. He used 112 of these images, or 0.05% of the images available in the scene, and ran the training on a mid-range laptop computer.

The trained model can take pixels from any spectral scene, infer the atmospheric transmission profile, and perform atmospheric correction automatically. At the core of this approach is a set of differential equations that describe how sunlight changes as it passes through the atmosphere, reflects off a target, and rises back up through the atmosphere to hit the sensor.

“The differential equation constraints, the physics-based machine learning, are the secret to ensuring that this works well,” Koch says. “By construction, what the model can make are predictions that satisfy first-order physics.”

In addition to its performance ranging between off-the-shelf models and high-fidelity approaches, Koch's framework is bidirectional, capable of removing atmospheric effects from spectral scenes collected by remote sensors and inferring what materials on the ground would look like when imaged through a particular atmosphere.

“Depending on where you look, some things will be highlighted or hidden,” Koch explained. “It's not a one-stop shop. You have to dig in and explore where it's most fruitful.”

Bringing research to the real world

Remote sensing is used for a wide range of tasks, from drought and vegetation indices that track changes in photosynthetic activity and water content over time, to detecting methane gas plumes, activity at foreign military bases, and human traffic at border crossings.

Different atmospheric correction approaches are applied to different scenarios depending on factors such as time, cost and available data.

Luis Cedillo, a PNNL intern and undergraduate student at the University of Texas at El Paso, presented a conference poster on using physics-based machine learning techniques for coastal ecosystem health monitoring at SPIE Defense and Commercial Sensing 2024 in National Harbor, Md. He used a machine learning pipeline to jointly learn atmospheric and coastal water profiles, enabling new capabilities to track coral reef health from satellites.

The researchers are now refining their approach, with an eye towards applications such as target detection, where data is limited but high fidelity is required.

“The main advantage here is that we can get high accuracy with a limited amount of data, without relying too much on prior knowledge of things like sensor location or the position of the sun,” Koch says. “We're learning that information in real time.”

“I've taken some of the things that experts in this field do and put it into a machine learning pipeline so that we can run that process in a data-driven way,” Koch said. “It's a compromise when you want higher fidelity but don't necessarily have the resources to identify all the relevant properties of the atmosphere. We use the data that's available.”