How do you train a robot to wash the dishes? This is a simple task, but training it to understand all the different small tasks within it is very complex.
A new method developed by researchers at Carnegie Mellon University shows a very simple strategy. Have your robot watch tons of videos. you are doing.
Innovative robot learning technology: In a recent paper, researchers found how to use two robots to successfully perform 12 household tasks, including taking pots off the stove, opening drawers and cabinets, and interacting with produce and objects such as knives and trash cans. I explained how I was trained.
a modern robot can perform these tasks. However, the characteristics of the new approach are: how The robot learned its behavior by watching videos of humans manipulating objects in the kitchen.
Traditional strategies require humans to manually program robots to perform specific actions or train them in simulated environments before trying out new tasks in the real world. Both strategies can be slow and unreliable.
The new technique, used in previous research by the Carnegie Mellon University team, trains robots to learn tasks by watching humans perform tasks in a controlled environment. The new method does not require a human to model the behavior in front of the robot. Instead, the robot “sees” an egocentric video (i.e., a camera on its forehead) of a human performing a specific task, and uses that information to begin practicing the task. They learned how to do it successfully in about 25 minutes.
To train the robot, the team developed a model based on the concept of ‘affordances’. This is a psychological term that refers to what the environment offers us in terms of possible behaviors. For example, a frying pan handle makes it easier to lift the frying pan.
By watching videos of humans doing household chores, the robots were able to recognize patterns in how humans were more likely to interact with certain objects. It then used that information to predict which set of actions it would need to perform to perform a given task.
The research team reported that their method, tested on two robots for over 200 hours in a “playing kitchen” environment, outperformed previous methods for real-world robotic tasks.
It’s worth noting that robots weren’t just watching people do things over and over again. same rather than the kitchen a lot of different Kitchen – Each may have different looking pots and drawers. In other words, the robot learned to generalize new skills.
It’s a capability made possible by advances in computer vision.
Computer vision: When training an AI to perform a task, it is much easier to train a language processing system like ChatGPT to write an essay than to train a robot to wash dishes. One reason is that it’s difficult to move robots properly in physical space, especially in new spaces. After all, not all kitchens are the same.
But there is also the problem of computer vision, the field of artificial intelligence that trains computers to “see.” Training computers to derive meaningful information from images has proven very difficult for decades.
First, let’s consider what the human process involves. For example, when you walk into a kitchen, it takes a second to visually recognize the sink, stove, refrigerator, cabinets, drawers, and other tools and appliances. Not only do we recognize these objects properly, but we also recognize how to use them and recognize their affordances.
This skill does not come naturally to a computer that initially “sees” the thousands of individual pixels that form the image of the kitchen as a whole. AI researchers need to train computers to recognize pixel-by-pixel patterns by being exposed to large numbers of labeled images, and eventually learn to distinguish between stoves and sinks, for example. (Remember her CAPTCHA, “Prove you’re not a robot and choose an image on the school bus”? You’re the one training algorithms like this.)
Still, correctly recognizing objects is only part of the battle.
“[E]Even with perfect perception, it’s hard to know what to do,” the researchers wrote. “Given an image, current computer vision approaches can label most objects and even give approximate information about where they are, but this is not the way a robot can perform a task. It also requires knowing where and how to manipulate objects, and figuring this out from scratch each time in a new environment is virtually impossible for all but the simplest tasks. is.”
The affordance-based method, which the researchers call a “vision robotics bridge,” helps robots learn simple tasks primarily by minimizing the amount of training data researchers need to provide to the robot. Help streamline your methods. In a recent study, training data was obtained from a database of egocentric videos.
But Sikar Baal, lead author of the study, suggested in a press release that robots may soon be able to learn from a wide range of videos on the internet. But that may require computer vision models that don’t rely on egocentric perspectives.
“We are using these datasets in new and different ways,” says Barr. “This research could enable robots to learn from the vast amount of internet and YouTube videos available.”
