Abilion several machines Read our thought has progressed steadily in recent years. Researchers are now using AI video generation technology to give us a window into our mind’s eye.
The main impetus behind attempts to interpret brain signals is the hope that people in comas and various forms of paralysis may one day be able to provide new windows of communication. But there are also hopes that the technology could create a more intuitive interface between humans and machines and could be applied to healthy people.
Most research so far has focused on efforts to recreate internal monologues.s We use an AI system to extract what words the patient is thinking. The most promising results have come from invasive brain implants, which are unlikely to be a practical approach for most people.
But now, researchers at the National University of Singapore and the Chinese University of Hong Kong have shown that non-invasive brain scanning and AI image generation techniques can be combined to create short video snippets that look strikingly similar to clips that subjects were watching. Indicated. when their brain data was collected.
This work is an extension of work by the same author published late last yearThere, they showed that they could generate still images that closely matched the photos shown to the subjects. This he achieved by training one model on a large amount of data that was first collected using her fMRI brain scanner. We then combined this model with the open-source image generation AI Stable Diffusion to create images.
in a new paper Published in preprint server arXiv, The authors take a similar approach, but adapt it so that the system can interpret streams of brain data and convert them into moving images instead of still images. First, they trained one of his models on a large number of fMRIs so that it could learn common features of these brain scans. This was then extended to handle a series of fMRI scans rather than individual fMRI scans, and was retrained on a combination of fMRI scans, video snippets that elicit their brain activity, and text descriptions.
Separately, the researchers adapted a pre-trained stable diffusion model to generate videos instead of still images. It was then trained again with the same video and text descriptions that the first model was trained with. Finally, we combined the two models and fine-tuned the fMRI scans and their associated videos.
The resulting system was able to acquire fresh, never-before-seen fMRI scans and produce videos that closely resemble clips taken by human subjects.d I was watching at the time. While far from a perfect match, the AI output was generally fairly close to the original video, accurately reproducing crowd scenes and horse herds, and often matching the color palette.
To evaluate the system, the researchers used a video classifier designed to assess how well the model understood the semantics of the scene. For example, did the model perceive the video as a fish swimming in an aquarium or a family walking down the street? Even if the image is slightly different. Their model scores 85%, which is 45% better than the state-of-the-art.
AI-generated videos still have their flaws, but the authors say this line of research could ultimately have applications in both basic neuroscience and future brain-machine interfaces. . However, they also acknowledge the potential drawbacks of this technology. “Ensuring the privacy of individuals’ biological data and avoiding malicious use of this technology will require government regulation and the efforts of the research community,” they wrote.
This is perhaps a nod to concerns that a combination of AI brain-scanning techniques could allow people to invasively record the thoughts of others without their consent. aI was worried again Earlier this year, the voice came up when researchers used a similar approach to essentially create a rough. transcription of voices in people’s headshowever, experts point out that this will result in: impractical if not impossible for the near future.
But whether you see this as an eerie invasion of privacy or an exciting new way to work with technology, machine mind readers seem to be getting a little closer to reality.
Image credit: Claudia Dewald from Pixabay
