Researchers at Aalto University have developed a revolutionary bio-inspired sensor that can detect moving objects in a single video frame and accurately predict their future motion.explained in Nature Communications According to the paper, this advanced sensor has numerous potential applications in areas such as dynamic vision sensing, automated inspection, industrial process control, robot guidance, and autonomous driving technology.
Traditional motion detection systems require a large number of components and complex algorithms to perform frame-by-frame analysis, making them inefficient and energy-consuming. To address these limitations, the Aalto University team looked to the human visual system for inspiration, and neuromorphics to integrate sensing, memory, and processing into a single device that can detect movement and predict trajectory. Created vision technology.
Photomemristor: the core of new technology
The researchers’ technology is built on an array of photomemristors, electrical devices that generate electrical current in response to light. The photomemristor has a unique property that the current does not stop immediately even when the light is turned off, but gradually decays. This feature allows photomemristors to effectively “remember” recent exposures to light, and sensors made up of arrays of these devices can provide not only instantaneous information about a scene, but also previous Momentary dynamic memory can also be captured.
“A unique property of our technique is the ability to integrate a series of optical images into a single frame,” explains Hongwei Tan, a research fellow who led the study. The information of each image is embedded in the following images as hidden information. In other words, the last frame of the video also contains information about all previous frames. This allows us to detect motion early in the video by using a simple artificial neural network to analyze only the last frame. The result is a compact and efficient sensing unit. ”
Demonstration of technical capabilities
To showcase the technology, the researchers used a video showing the letters of words one by one. All words ended with the letter ‘E’, but traditional vision sensors couldn’t discern if the ‘E’ on the screen was followed by any other letter in ‘APPLE’ or ‘GRAPE’ . However, the photomemristor array was able to use the information hidden in the final frame to guess which letter was before it, predicting the word with near 100% accuracy.
In another experiment, the team showed sensor videos of a simulated person moving at three different speeds. The system was not only able to analyze a single frame to recognize motion, but also accurately predicted subsequent frames.
Impact on autonomous vehicles and intelligent traffic
Accurate motion detection and trajectory prediction are critical for autonomous driving technology and intelligent transportation systems. Self-driving cars make informed decisions by accurately predicting how cars, bicycles, pedestrians, and other objects will behave. By incorporating a machine learning system into the photomemristor array, the researchers demonstrated that the integrated system can predict future motion based on in-sensor processing of all information-bearing frames.
“Action recognition and prediction with our compact in-sensor memory and computing solutions offers new opportunities for autonomous robotics and human-machine interaction,” said Prof. Sebastiaan van Dijken. “The in-frame information captured by the system using Photomemristor avoids redundant data flows and enables real-time, energy-efficient decision-making.”
