“Alexa, play Kirsten’s dream I saw last week.” could be a command.
Researchers at the National University of Singapore and the Chinese University of Hong Kong reported last week that they had developed a process that could generate videos from brain scans. This research arXiv preprint server.
Researchers Jiaxin Qing, Zijiao Chen, and Juan Helen Zhou use a process called functional magnetic resonance imaging (fMRI) to combine the data acquired by imaging with a deep learning model, Stable Diffusion, to create a smooth created a high quality video.
Successful reconstruction of still images collected from brain activity through AI-assisted stable diffusion, commonly used for image generation, has been reported in recent months. However, as Qing and his team reported, acquiring continuous visual images poses special challenges.
The fMRI process typically utilizes blood oxygenation level dependent (BOLD) signals to acquire images of brain activity every few seconds. This will produce a low quality video image. The standard video capture rate is 30 frames per second.
“Understanding the information hidden in our complex brain activity is a great mystery in cognitive neuroscience,” says Chin. “Especially for him, using non-invasive tools like fMRI to recreate human vision from brain recordings is an exciting but challenging task.”
His team used the Mind-Video model to achieve high quality video. Described as “his two-module pipeline designed to bridge the gap between image and video brain decoding,” his fMRI decoder gradually learns from acquired brain signals, Training and fine-tuning using an image database.
The result, they say, is high-quality video with motion and scene dynamics with 85% accuracy.
Qing says their work shows promise for future large-scale modeling applications “from neuroscience to brain-computer interfaces.”
Combining AI with MRI and EGM (electromyography) to study images, brain activity, and muscle movement is opening up new perspectives on how the mind works. Shinji Nishimoto, a neuroscientist at Osaka University, says these processes could one day be used to capture thoughts and dreams.
Dream researcher Daniel Ordis, working with colleagues in the Cognitive Neuroscience Lab at the University of Texas at Austin, tracks brain activity and muscle nerve impulses to define images, sounds and movements in dreams. He is working on the development of dream recording MRI technology.
“This is like the early days of the space race,” he said recently. “But in this case, we enter the dream space.”
The average person has up to six dreams a night, but forgets 90% of them within minutes of waking up. It’s so much fun to be able to record your dreams forever.
Dreams have puzzled us for centuries. Early cultures believed that dreams were messages from the gods. In the 20th century, Freud introduced the idea that behind our dreams are repressed sexual and emotional urges.
Some of our greatest inspiration comes from the nocturnal nights we all spend every night of our lives.
The melody of The Beatles’ “Yesterday,” voted the best song of the 20th century in a BBC radio poll, came to Paul McCartney in a dream. The famous three-note opening riff to the Rolling Stones’ megahit “Satisfaction” came to Keith Richards in the middle of the night. He woke up to record a rough version of the song, sleepily muttered, “I’m not satisfied,” and went back to sleep, listening to the tape that morning for the first time to realize what he had written down. Noticed.
Salvador Dali called his fantastic surrealist works, such as The Persistence of Memory, “hand-painted dream pictures.”
Also, a nervous student, worried that his admission to Stanford University was a mistake, had a disturbing dream of downloading the entire web onto his computer. When he woke up, he set out on a project that one day would attract 89 billion monthly viewers and he would answer 99,000 inquiries per second. His name is Larry Page. His creation is Google.
From nightmares to game-changing inspirational ideas, recording and streaming our dreams may just be a dream for now.
However, as the works of Kiyo and Ordis show, it did not last long.
For more information:
Zijiao Chen et al., Cinematic Mindscapes: High-Quality Video Reconstruction from Brain Activity, arXiv (2023). DOI: 10.48550/arxiv.2305.11675
Magazine information:
arXiv
© 2023 Science X Network
