Researchers at UCLA and the US Army Research Laboratory have launched a new approach to augment computer vision technology with artificial intelligence by adding physics-based perception to data-driven technology.
was announced in nature machine intelligenceThis study outlines a hybrid approach designed to improve how AI-based machines sense, interact with, and respond to their environment in real time. For example, how self-driving cars move and steer, or how robots use improved technology to perform precision. action.
Computer vision enables AI to see and understand its surroundings by deciphering data and inferring properties of the physical world from images. Such images are formed through the physics of light and mechanics, but traditional computer vision techniques have primarily focused on database machine learning to improve performance. Physics-based research has been developed in a different trajectory to explore the different physical principles behind many challenges in computer vision.
Incorporating an understanding of physics (the laws that govern mass, motion, etc.) into the development of neural networks has been difficult. In neural networks, AI modeled after the human brain with billions of nodes processes massive image data sets to understand what they “see.” However, there are now several promising lines of research that seek to add an element of physical perception to already robust data-driven networks.
“Physics-aware forms of reasoning will enable safer driving of cars and more precise driving of surgical robots,” said Achuta Kadambi.
UCLA research aims to harness the power of both deep knowledge from data and the real-world know-how of physics to create hybrid AI with enhanced capabilities.
“Visual machines (cars, robots, health appliances, etc. that use images to perceive the world) ultimately perform tasks in our physical world,” said the study’s corresponding author. Achuta Kadambi, Assistant Professor of Electrical and Computer Engineering, UCLA Samueli School of Engineering. “Physics-aware forms of reasoning will make it possible to drive cars more safely or drive surgical robots more accurately.”
The research team outlined three ways physics and data are beginning to combine into computer vision artificial intelligence.
- Incorporating Physics into AI Datasets
Similar to characters in video games, objects are tagged with additional information such as movement speed and weight.
- Incorporating physics into your network architecture
Run the data through a network filter that encodes physical properties to encode what the camera picks up.
- Incorporating Physics into the Network Loss Function
Leverage knowledge built on physics to help AI interpret training data about what it observes
These three lines of research are already yielding encouraging results in improving computer vision. For example, a hybrid approach will allow AI to more accurately track and predict the movement of objects, producing accurate high-resolution images from scenes covered by bad weather.
With continued progress in this dual-modality approach, the researchers say, deep-learning-based AI could begin to learn the laws of physics on its own.
Other authors on the paper are Army Research Laboratory computer scientist Celso de Melo and UCLA computer science professor Stefano Soatto. His Cho-Jui Hsieh, Associate Professor of Computer Science, and Mani Srivastava, Professor of Electrical Engineering and Computer Engineering and Computer Science.
This research was supported in part by a grant from the Army Research Laboratory. Kadambi is supported by grants from the National Science Foundation, the Army Young Investigator Program, and the Defense Advanced Research Projects Agency. Vayu Robotics co-founder Kadambi is also funded by his Intrinsic at Alphabet. Hsieh, Srivastava and Soatto He is supported by Amazon.
