When we learn a new skill, the brain has to decide what to change, cell by cell. New research from MIT suggests that it can do so with surprising precision and send targeted feedback to individual neurons, allowing each neuron to adjust its activity in the right direction.
This discovery reflects an important idea in modern artificial intelligence. Many AI systems learn by comparing their output to a target, calculating an “error” signal, and using it to fine-tune connections within the network. A long-standing question is whether the brain also uses such individualized feedback. In an open access study published in the February 25 issue of the same journal, natureMIT researchers report evidence that this is the case.
A research team led by Mark Harnett, a researcher at the McGovern Institute for Brain Research and an associate professor in the Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology, discovered these cue signals in mice by training the animals to control the activity of specific neurons using a brain-computer interface (BCI). According to the researchers, their approach can be used to further study the relationship between artificial neural networks and the real brain, and is expected to improve our understanding of biological learning and enable better brain-inspired artificial intelligence.
changing brain
Our brains constantly change as we interact with the world, changing our brain circuits as we learn and adapt. “Fifty years of research has shown us that there are many ways to change the strength of connections between neurons,” Harnett says. “What’s really missing in this field is a way to understand how these changes are coordinated to actually produce effective learning.”
Some behaviors and the neural connections that make them possible are enhanced by the release of neuromodulators such as dopamine and norepinephrine in the brain. However, these signals are broadcast to large groups of neurons without distinguishing between cells’ individual contributions to failure or success. “Reinforcement learning with neuromodulators works, but it’s inefficient because every neuron and every synapse essentially only receives one signal,” Harnett says.
Machine learning uses a very powerful alternative method for learning from mistakes. Using a method called backpropagation, artificial neural networks calculate an error signal and use it to adjust individual connections. They do this over and over again, learning from experience how to fine-tune their networks for success. “It works very well and is computationally very efficient,” Harnett said.
It is likely that the brain also uses similar error signals for learning. But neuroscientists were skeptical that the brain could precisely send tailored signals to individual neurons because of the constraints imposed by using living cells and circuits instead of software and equations. The main problem in testing this idea was how to find signals that give individual instructions to neurons, called vectorized instruction signals. Valerio Francioni, lead author of the book, explains this challenge: nature The paper and the former postdoc in Harnett’s lab say scientists don’t know how individual neurons contribute to specific behaviors.
“If I was recording your brain activity while you were learning to play the piano, I would see a correlation between the changes happening in your brain and the fact that you are learning to play the piano. But if you asked me to manipulate your brain activity to make you a better piano player, I wouldn’t be able to do that because I wouldn’t know how the activity of individual neurons maps to its ultimate performance,” Francioni explains.
Without knowing which neurons need to be made more active and which need to be suppressed, it is impossible to look for the signals that direct those changes.
Understand neuron function
To get around this problem, Harnett’s team developed a brain-computer interface task that directly links neural activity to reward outcomes. This is similar to directly linking the piano keys to the activity of a single neuron. To succeed in this task, the activity of certain neurons must be increased while the activity of others must be decreased.
They set up a BCI that directly linked the activity of just eight to 10 neurons out of the millions in the mouse brain to visual readouts, providing the mice with sensory feedback about their performance. Success came with sweet rewards.
“So when you ask me, ‘How can mice get more reward? Which neurons have to be activated and which have to be inhibited?’ I know exactly what the answer to that question is,” Francioni says. His research was supported by a Y. Eva Tan Fellowship from the Yan Tan Collective at MIT.
Although scientists did not know the exact function of the specific neurons linked to the BCI, the cells were active enough that the mice could occasionally receive a reward if the signal happened to be correct. Within a week, the mice learned how to turn on the correct neurons and keep other sets of neurons inactive, earning them more rewards.
Dr. Francioni monitored the target neurons daily during this learning process using a powerful microscope to visualize fluorescent indicators of neural activity. He focused on the branching dendrites of neurons, where it has long been suspected that the appropriate feedback signals arrive. At the same time, they tracked the activity of the parent cell bodies of those neurons. The research team used these data to examine the relationship between the signals received in the neurons’ dendrites and their activity, and how the signals change when the mouse activates the appropriate neurons and is rewarded, or when the mouse fails a task.
vectorized neural signals
They concluded that the two neuron groups whose activity controlled BCI in opposite ways also received opposing error signals in their dendrites as the mice learned. Some were instructed to increase their activity during the task, while others were instructed to decrease their activity. Furthermore, when the researchers manipulated the dendrites to suppress these instructional signals, the mice were unable to learn the task. “This is the first biological evidence of vectorization. [neuron-specific] “Signal-based instructional learning takes place in the cortex,” Harnett said.
The discovery of vectored signals in the brain, and the team’s ability to find them, should facilitate further interaction between neuroscientists and machine learning researchers, said postdoctoral researcher Vincent Tang. “This provides further incentive for the machine learning community to continue developing models and proposing new hypotheses along this direction,” he says. “Then you can come back and test.”
The researchers say they are as excited about applying their approach to future experiments as they are about the current findings.
“Machine learning provides a robust and mathematically tractable way to study learning in practice, and the fact that we can now translate at least some of this directly to the brain is extremely powerful,” Francioni says.
Harnett says this approach opens new opportunities to explore possible similarities between the brain and machine learning. “How does the cortex learn? How do other brain regions learn? How similar or different is it from this particular algorithm? Can we find ways to build better, more brain-inspired models based on what we learn from biology?” he says. “This really feels like a big new beginning.”
