How does the human brain deal with complex situations, such as driving through traffic in Harvard Square at 5pm?
One popular theory among psychologists and neuroscientists is that the brain creates causal models of the world that help us plan and act. It’s like running a simulation in your head to see which results are good or bad. “It can be used to learn internal models of the environment and predict what will happen if we take different actions,” said Momchir Tomov, an associate professor of psychology in Professor Samuel Gershman’s Computational Cognitive Neuroscience Laboratory. I can,” he explained..
Over the last few decades, computer scientists have developed these ideas into a system called Reinforcement Learning, or RL for short. Researchers like Tomov, who work at the intersection of psychology and technology, have also introduced computational models that try to capture how RL works in the brain. In a new paper published in Neuron, Tomov and his co-authors compare algorithmic theory to real-world imaging using functional magnetic resonance (fMRI)..
Why create algorithms that attempt to formalize human thinking and decision-making? “It’s difficult to study cognitive processes without precise computational models that map inputs to outputs,” says Ph.D. Acquired Mr. Tomov says. He received a PhD in neurobiology from Harvard University in 2019 and worked with Gershman as a postdoc until 2021.
The researchers also hope their work will lead to advances in RL, which can navigate complex environments and is considered one of the greatest success stories in artificial intelligence. In fact, they outperform humans in areas such as board games and video games, but until recently have proven to be somewhat slower learners. “A more human-like algorithm will perform better than traditional machine learning in certain areas,” Tomov said.
The group’s experiment builds on previous work by two of the study’s co-authors. In 2021, Thomas Pouncey, another postdoctoral fellow in the Gershman lab, outlined a RL system based on more complex theory. His computational theory-based RL model was introduced in a subsequent paper by MIT Postdoctoral Fellow Pedro Tsividis. It has proven to be much faster than previous iterations in learning new video games. In terms of speed, Tomov said it’s much closer to human ability to handle such tasks.
Throughout this process, researchers made hypotheses about the neural architecture of human decision-making and learning. In a new study, researchers tested the algorithm on 32 volunteers who eventually mastered playing an Atari-style video game while connected to an fMRI scanner that measures small changes in blood flow that accompany brain activity. bottom.
As the researchers expected, this provides evidence for a theory-based model of activity in the frontal prefrontal cortex of the brain, and evidence that theory updating occurs in the posterior cortex, the back of the brain. rice field. It was in the details where their hypotheses and algorithms diverged. The researchers particularly hoped to find evidence for a theoretical model in the orbitofrontal cortex. Instead, they found them in the inferior frontal gyrus. This makes sense in hindsight, because previous work by Gershman’s lab has found that the inferior frontal gyrus is involved in learning “the laws of causality that govern the world,” says Tomov. said Mr.
Even more surprising was the finding in the back of the brain. There, the occipital cortex and ventral pathways (both central to visual processing) appear to be involved when the model needs to be updated. “Whenever we get surprising information that contradicts the current theory, then we not only see an update signal in the ventral pathway, but also when that theory is activated in the inferior frontal gyrus,” says Tomov. summed up Mr.
Finally, fMRI scans revealed the direction of information flow in the brain. Tomov and his co-authors hypothesized that information flows bottom-up. Instead, it appears top-down and flowing during gameplay.
“It’s as if it’s emanating from a model stored somewhere in the prefrontal cortex and running down into the posterior visual area,” he said. “But when a mismatch occurs, that is, when an update occurs, the pattern of information flow is reversed. Information now flows bottom-up from the posterior region to the anterior region.”
Tomov has been studying theory-based RL for four years under Gershman. Two years ago, as a full-time employee at a Boston venture, he began applying these ideas to self-driving cars. “How can I turn left from here to the next intersection without hitting anyone?” he asked. “There is basically an internal model of the world that predicts other drivers and what they are going to do.”
