When a deep learning AI model accurately predicts tomorrow’s river flow, do we understand the reasoning behind it? That’s what Ala Bayati is trying to find out.
Bayati, a PhD candidate in the Department of Earth, Ocean and Atmospheric Sciences, was initially interested in how nature supports us and the complexities of nature, which is both “kind and cruel”, being kind because of the resources it provides, but cruel with the floods and disasters that can follow. The importance of nature to human life led him to study water resource management.
“Water security is one of the pillars of security for any country,” Bayati said in an interview. Ubissey. Water is linked to food systems and economic stability. This means that predicting river flows is not only about safety, but also about understanding water behavior for our own use.
Deep learning is increasingly being used in rainfall-runoff modeling, a process that predicts the amount of water flowing into rivers based on hydrological data and simulations of the water cycle. However, Bayati and his co-authors point out in a new paper that while models such as long-term short-term memory (LSTM) networks can achieve high accuracy, there are pitfalls. These models learn statistical patterns rather than visible hydrological mechanisms, raising questions about whether their internal logic is reliable.
Accuracy and reliability are not the same. Bayati compares this difference to studying for an exam. You may do well if you memorize exactly what is on the test (accuracy). But when you change the question to require deeper understanding and application, you fail (lack of credibility). “Testing” water forecasts is complicated by climate change, heatwaves, and unknown extreme events.
This problem (of models not being able to pass harder tests) is rooted in how most machine learning works. Deep learning models are often described as “black boxes.” That is, you know what’s going in and what’s coming out, but you don’t know what’s happening in between.
In a river flow model, the input is environmental data. Bayati’s paper considers daily precipitation (rain/snow), temperature, evaporation, and observed river flows from 1,110 watersheds (areas where water naturally collects) in North America. The output is river flow prediction. But when a model is a black box, Bayati says, you don’t actually know how its predictions were made or whether they are made in a physically meaningful way. “I don’t know if it’s just…memorization.”
To address this issue and test whether these “high-performance” models are functionally realistic (i.e., whether the inner workings and processes of the models are consistent with defensible physical mechanisms), the team used the Explainable AI (XAI) framework. XAI is inherently “open”[s] Use this “black box” to peek inside and see if a model that looks accurate on paper actually uses sensible relationships.
The team didn’t focus on where the model was performing poorly. The real question was whether high accuracy could be obtained from correct inference.
The answer was no. Mr Bayati said that across the 672 catchments they assessed, “we saw bad inferences all over the place”.
In rainy basins like the West Coast, the model behaved as if heat alone would increase flow. “Temperature should not affect river flows in areas with a lot of rain,” Bayati said, but short-term increases in temperature led to an increase in simulated river flows.
Similarly, in areas with a lot of snow, the model tended to misattribute flow sources related to snowmelt. Simply put, the model recognized that increased evaporation and increased river flow tend to occur together, and interpreted this as a causal relationship where increased evaporation causes increased river flow. The model missed a third variable that actually causes increases in both evaporation and river flow: temperature.
This model confused “things that happen together” and “things that cause each other.” This is something that every research student knows is wrong; correlation is not causation. However, this fault poses risks when making predictions under new climate conditions. The authors argue that this correlation-driven learning can compromise accuracy under extreme weather conditions and short- and long-term climate change.
The natural follow-up is: If you know the physics, why don’t you add it? In fact, the paper describes physics-based deep learning models as a way to add constraints to prevent the model from violating relationships that could never occur physically. However, although these physics-backed models exist, they are not widely used for two reasons.
First, it’s convenience. Pure deep learning is much easier to train and deploy. Second, trust: Although physics-based methods are still new in some subfields, such as hydrology, and some researchers remain unsure whether they can deliver on their promise, Bayati said the literature is growing and he himself is working on follow-up studies of physics-based models.
Bayati’s research doesn’t claim that deep learning can’t help hydrology, but it does show that mere performance is not enough.
As Vancouver approaches its first snow-free winter since 1982, risks are rising as the climate becomes less predictable and water systems are forced into new conditions, according to Environment Canada. And the model’s predictions need to hold up as the rules change. “Maybe in the future, [they’ll be] Much better. ”
