A team of Georgia Tech researchers has developed a new machine learning framework that allows humanoid robots to walk on sand, gravel, wet grass, slopes, stairs, and slippery surfaces while reducing the time and computational power required to train controllers.
The researchers say their approach, called “Learn to Teach,” improves on the popular teacher-student reinforcement learning method by training both agents simultaneously instead of sequentially. The result is a controller that can handle untrained terrain while reducing the required computational resources.
The controller was tested on a bipedal humanoid robot and successfully traversed a variety of difficult outdoor and indoor surfaces. During the experiment, the researchers pushed and pulled the robot, and it adjusted its gait to keep it stable.
The study was presented at the IEEE International Conference on Robotics and Automation (ICRA), where the researchers described a training framework that can be applied to other robots and tasks beyond walking.
teach while learning
Traditional teacher-student reinforcement learning relies on initially creating a “teacher” model that has access to detailed simulation data. Once fully trained, the teacher passes that knowledge to a “student” model that controls the actual robot.
According to lead researcher Feiyang Wu, this process has two major drawbacks.
“There are two problems with this approach: It takes too long to train sequentially, and a lot of the information collected by the teacher is wasted.”Training a robot controller through simulation requires hours of computation on expensive GPU hardware, making the process slow and expensive.
Rather than waiting for teachers to master the task, the Georgia Tech team trained teachers and students together. Once the teacher learned gradually, he immediately began to transfer knowledge to the students, which significantly shortened the training process.
“Teachers don’t have to wait until they become experts to start teaching students,” Wu said. “Teachers can gradually teach students what they learn along the way.”
The researchers also allowed teachers to learn from their students’ experiences. This reduced what roboticists call the “teacher-student imitation gap,” where students encounter situations that differ from the teacher’s ideal simulation.
real terrain success
The new controller was installed on a life-sized humanoid robot in Associate Professor Ye Zhao’s laboratory. It was able to move across rough outdoor terrain and slippery indoor surfaces without relying on separate controllers for different environments.
Wu said the team did not expect the same controller to perform so well under so many conditions. “For this large, very tall humanoid robot, it hasn’t really been proven that it can do agile locomotion in such tough terrain. Somehow, the highly efficient training recipe here actually works in all types of terrain and environments.”
Chao said the controller performed better than software provided by the robot’s manufacturer, demonstrating the value of combining machine learning research with real-world robotics. Beyond humanoid locomotion, researchers believe the “Learn to Teach” framework can also be applied to other robot designs and tasks that require reliable behavior in unpredictable environments.
This research IEEE International Conference on Robotics and Automation.
