Scientists used artificial intelligence (AI) to read the brains of mice watching video clips and reconstruct what they saw.
Scientists at the Swiss Federal Institute of Technology (EPFL) have developed a new machine-learning algorithm that converts brain signals into video.
Called CEBRA (pronounced zebra), the algorithm reveals hidden structures in data recorded from the brain and has the potential to predict complex information.
In experiments, the researchers used the new technique to reconstruct the films seen by mice.
“We asked the question: Can we really reconstruct what the animals saw purely from neural data?” says EPFL neuroscientist Mackenzie Mathis.
“We used a new algorithm, CEBRA, to build a latent representation of this embedding space. We then took this embedding space and basically used it as the basis for our neural decoding algorithm, which we used to determine what the mouse is looking at. It is possible to accurately predict the sequence of frames with
The researchers used CEBRA to map brain signals and movie features from brain data recorded at the Allen Institute in Seattle, Washington.
Researchers in Washington showed mice a black-and-white movie clip of a man running to a car and opening the trunk.
Mouse brain signals were measured via electrode probes inserted into the visual cortex region. Optical probes were used in genetically engineered mice to make the mice’s neurons glow green when transmitting information while the mice were passively watching movies.
Experts say it’s a common procedure for genetic engineering amid growing interest in brain research.
“There’s a technique called optogenetics that uses genetic markers that are bred to mice so[…]they have no effect on the mice,” says Nadia Rosen, scientific director and professor at the Jackson Institute. Dr. Tarr said. For mammalian genetics.
“The mice don’t even know they’re there. We can track when the nerves are firing. So we can actually see the firing network of the nerves in the brain.” she told Euronews Next.
Mind-reading AI
CEBRA boasts high accuracy, and the AI-reconstructed video matched the original almost perfectly, albeit with minor distortions.
“Using this algorithm, we were able to do this with over 95% accuracy on these movies, so this is the first demonstration that this brain-machine interface-style decoding is actually possible. I think it’s kind of like,” Mattis said.
Other researchers around the world have recently made breakthroughs in using AI to decode brain signals. Just last week, a team in Austin, Texas, unveiled a system that can convert someone’s brain activity into a continuous stream of text.
Another study, conducted in March by a team at Osaka University in Japan, revealed how AI can read brain scans and recreate images seen by humans.
Although it’s not yet possible to completely reconstruct what humans see based on brain signals alone, the developers believe CEBRA has potential for clinical applications beyond neuroscience.
“It can be used for things like optic nerve prostheses, potentially restoring vision or allowing arm movement. So also for patients who are paralyzed and who want to recover or even strengthen in this way.” added Mattis.
Rosenthal agrees that technologies like CEBRA have great potential.
“We now have an amazing amount of new technology that allows us to observe what’s going on in the mouse brain,” said Rosenthal.
“Being able to develop these little markers that can track what’s going on without having to kill the mouse to open the brain is very helpful,” she added.
For more on this story, watch the video in the media player above.
