Three University of British Columbia students put their expertise to the test by building an artificial intelligence (AI)-controlled single-person air hockey table. That’s all interesting enough, but the real problem is that the AI was able to learn how to play purely in simulation.
Typically, such AI-controlled robots are trained in the “real world” through trial and error, failing and learning from thousands of practice runs. Although effective, this is time consuming and can damage your hardware.
But the UBC team decided to train the AI in a digital space, see if it makes mistakes there first, and then lets loose on a real air hockey table. To achieve this goal, they built a high-precision digital twin of the table to train the AI.
Once trained, the trained AI was copied to a real robot to see how it performs against human adversaries. And the results were amazing.
Despite their real-world experience, the AI-controlled robot air hockey players were able to pose a genuine challenge. All of these are “out of the box” so to speak.
Digitally tailored to the real world
What’s also interesting here is that AI typically has a hard time learning tasks like air hockey. This is especially true because the puck can move very quickly and unpredictably, and is highly influenced by subtle changes from interaction with the paddle and the wall.
Robotic players must also consider time lag due to motors, cameras, voltage fluctuations, mechanical vibrations, and imperfect puck tracking. Therefore, even the smallest mistake can lead to a miscalculation that will cost you the game.
To achieve this goal, the team intentionally designed the AI training environment to be imperfect. This is a departure from most training scenarios and may be “too perfect”, but the real world is clearly not like that.
So the team factored in things like uneven rails, warped tables, inconsistent rebounds, power drops, and camera lag. This, called “domain randomization,” allows the AI to learn to anticipate the unexpected and respond as best it can, just as human players do.
The team explains that this teaches the AI to predict different possible outcomes from puck bounces, rather than the exact outcome based on a set algorithm. The AI has therefore learned to predict the approximate future location of the puck and react accordingly.
To speed up training, the team ditched the idea of using regular physics engines like Unity or Unreal, and instead turned to something called “soft actor critical” training. In short, this is a kind of learning environment based on carrots and sticks.
Air hockey is a punishable sport
In it, the AI performs actions and receives rewards or punishments based on its performance. Over millions of simulated games, the AI becomes better and better at adapting to game play and all the chaos built into it.
Applied to the real world, the AI players also provided assistance using special overhead cameras and pucks coated with retro-reflective tape. This allows the AI to “see” the puck more clearly at 120 frames per second.
Beyond air hockey, the team’s innovations also have potential applications in training other autonomous systems such as drones, vehicles, and robots. Similar simulation-to-realistic transfers could greatly benefit as training on these platforms becomes faster and more realistic.
If you want to learn more, you can find more information on the project’s official GitHub page.
