Research suggests that structured exploration allows biological brains to learn faster than AI

Neuroscientists have shown how exploratory behavior allows animals to learn their spatial environment more efficiently. Their findings could help build better AI agents that learn faster and require less experience.

Researchers at UCL’s Sainsbury Wellcome Center and Gatsby Computational Neuroscience Unit have found that the instinctive exploratory executions performed by animals are not random. These deliberate actions enable mice to learn world maps efficiently. The study, published today, April 28, neurondescribes how neuroscientists tested the hypothesis that certain exploratory behaviors performed by animals, such as quickly lunging toward an object, are important in learning how to navigate the environment. doing.

“There are many theories in psychology about how performing certain behaviors facilitates learning. This study explores whether simply observing obstacles in the environment is sufficient to learn about them. Or to test whether intentional sensory-induced behavior can help build cognitive abilities in animals, said Professor Tiago Blanco, group leader at the Sainsbury Wellcome Center and corresponding author of the paper.

In previous studies, SWC scientists observed a correlation between how well animals learned how to avoid obstacles and how many times they ran to them. In this study, Philip Shamash, a SWC Ph.D. student and lead author of the paper, conducted experiments to test the effects of preventing animals from performing exploratory executions. By expressing a light-activated protein called channelrhodopsin in parts of the motor cortex, Philip used optogenetic tools to prevent animals from launching exploratory runs toward obstacles. was completed.

The team found that even if mice spent a lot of time observing and sniffing for obstacles, they would not learn if they were prevented from running towards them. Exploratory behavior itself indicates that it helps animals learn the map of their environment.

The team worked with Ph.D. Sebastian Lee to investigate algorithms the brain might be using to learn. Students in Andrew Saxe’s lab at SWC ran different models of reinforcement learning that people have developed for artificial agents and observed which models best replicated mouse behavior.

There are two main classes of reinforcement learning models: model-free and model-based. The team found that under some conditions mice behave in a model-free manner, but under other conditions mice appear to have a model of the world. I implemented an agent that can arbitrate between and model-based. This isn’t necessarily how the mouse brain works, but it helped me understand what learning algorithms need to do to explain behavior.

“One of the problems with artificial intelligence is that an agent needs a lot of experience to learn something. One reason for this is that, unlike artificial agents, animal exploration is not random, but focused on salient objects. Directed exploration makes learning more efficient and requires less experience to learn,” explains Professor Blanco.

The next step for researchers is to investigate the link between exploratory behavior execution and subgoal representation. The team is now recording in the brain to discover the regions involved in subgoal expression and how exploratory behavior leads to expression formation.