Researchers have upgraded an algorithm called Daydreaming that allows neural networks to store and recall memories. It now works even when the training source data is highly biased.
For heavily distorted images, where these systems typically stumble, they maintained near-perfect reproduction even when previous methods failed.
Very little real-world data is evenly balanced, so the result is a kind of brain-inspired memory that comes much closer to the messy information that our eyes and cameras actually take in.
This is done using only simple local learning rules, of the kind that biologists consider to be realistic for living neurons.
How the network remembers
The system behind this research is the Hopfield network, one of the oldest and most studied models of how the brain retains memories.
It is built of simple units, loosely based on neurons, that turn on and off, pulling each other toward familiar patterns.
Each saved memory sits at the bottom of a kind of basin, so any partial or corrupted versions slide back into the complete pattern. By restoring the whole from fragments, the network becomes an associative memory.
“Every time we see a tree, our brain recalls the concept of a tree. This ability to associate many different representations with the same concept is called associative memory,” said Federico Ricci-Tersenghi, a theoretical physicist at Sapienza University of Rome.
The classic way of wiring such networks, the Hebb rule, ensures that only about 14 memories per 100 neurons are stored reliably before recall fails.
As more people gather, the network creates phantom states that don’t match real memories, reducing reliability.
learn like a dream
One way to clean up the mess is to borrow a trick from sleep.
In the offline phase, the network runs from a random starting point, settles into a formed phantom state, and then weakens that state.
Researchers call this process “unlearning” or “dreaming.”
These phantom states are false memories of the network, stable patterns that the network treats as real even though nothing is stored there.
Dreams exist to erase them, so remembering settles into real memories instead.
Drive away false memories with dreams
If done inadvertently, this cleanup can go too far, erasing real memories along with false ones, creating a problem known as catastrophic forgetting.
A 2025 study by Ricci-Tersenghi et al. introduced a modification called Daydreaming. It strengthens real memories and erases false memories in one continuous process.
“We combined daytime learning with the cleansing and consolidation stages of sleep, as if we were dreaming during the day,” Ricci-Tarsenghy said.
On balanced data, the network stored almost one memory per neuron, close to the theoretical limit, and also processed a database of handwritten digits.
real data breaks it down
All of this was based on one silent assumption: that the data was balanced. This balance is rarely maintained.
In a photo with a mostly bright sky or mostly dark shadows, nearly every pixel is driven to the same value, drowning out the small differences that distinguish one memory from another.
To see how severe the problem became, the team gradually increased the network imbalance by 500 units.
The leading traditional methods performed well on balanced data, but their recall steadily worsened as the skew increased.
In the most extreme settings, memories can only be reconstructed if the starting pattern was already nearly perfect.
The real data has an added twist. Its parts are rarely independent and tend to vary together in hidden, patterned ways.
This structure runs through most natural images and real-world signals.
focus on differences
This new study, led by Mikiya Doi of Tohoku University in Japan, changes what networks focus on.
Rather than storing the raw value of each unit, we store how far each unit is from the average of all memories.
“If you instead focus only on what changes compared to the average face, the differences become more apparent,” says Rich Tercenghhi.
Removing the shared background leaves only the distinctive parts of each pattern. This is exactly what you need to detach your memories.
The new version maintained the same noise tolerance throughout, even as the imbalance increased from uniform to highly skewed.
This completely reconstructed the memory from a rough starting point, breaking the old approach.
No manual adjustments were required. A single fixed configuration worked well enough.
In the landscape of memory
The reason why the centrally located version is better is because of its internal wiring.
The learned pattern of connection strengths spreads its influence over more internal directions than the old rule, with the single strongest direction eventually becoming more than twice as strong.
This wide spread sets up a kind of hierarchy between the directions.
Recollection seemed to rely first on the strongest, and only later on the weaker ones, and orders for separate research on the dreaming process were already beginning to be charted.
The team performed a clean test to confirm that widespread infection was indeed the culprit.
They manually rebuilt the connections and progressively flattened the range in that direction, leaving everything else unchanged, and watched the network’s noise tolerance shrink precisely as the spread narrowed.
The secret is in the wiring
What’s new is a brain-inspired memory that holds distorted structured data that previously corrupted it, and a clear reason for it.
It is not the details of the memorized pattern, but the broad expanse of its wiring that carves out a generous basin around each memory.
Learning stays local, touching only pairs of connected units rather than the entire network.
“It’s much more practical to make each decision locally,” Ricci-Tersenghi said.
The authors suggest that understanding how these models classify signal and noise could help build artificial intelligence that is easier to interpret and less energy-intensive.
This research Journal of Statistical Mechanics: Theory and Experiment.
—–
Like what you read? Subscribe to our newsletter for fascinating articles, exclusive content and the latest updates.
Check us out on EarthSnap, the free app from Eric Ralls and Earth.com.
—–
