AI gameplay still has flaws: testing AlphaZero-style self-play with Nim

• Researchers tested AlphaZero-style self-play with Nim, a mathematically solved children’s game.
• Despite rigorous training, agents have blind spots and can miss optimal moves.
• Good performance does not necessarily mean that the system has learned the underlying winning formula.

Embargo: Immediate.

New research published inmachine learningWe show that pattern learning alone is not enough to train AI to work on games, and that abstract representations and hybrid approaches can be helpful.

Many AI researchers describe gameplay as the “Formula 1” of AI. This is a controlled testing environment with clear rules and clear success criteria. In this paper, we use that idea as a diagnostic to study a matchstick game for children called Nim, a very simple game in which the optimal strategy is precisely known.

Since the correct move is known for every position, we can measure whether the agent is playing optimally across its state space. The study found that despite intense training and exploration, while small boards work, the agent has blind spots that can cause it to miss optimal moves, and as the board grows, performance decreases and predictions become more random. This suggests that fair games often require analytical representation rather than pattern learning.

What does this mean for machine-based games?
Self-playing AI is very powerful, but in games where both players share “pieces” and winning strategies are abstract arithmetic rules, pattern recognition from raw positions may not be enough.

Wider impact:
This result does not undermine the achievements of self-playing AI in games such as Chess and Go. Rather, they help map where today’s methods may have difficulties and where more abstract representations or hybrid approaches may be beneficial. More broadly, this is a reminder that while the system performs well in the common case, it remains vulnerable in rare but important cases.

Dr Soren Rees-Leeder, Professor of Computer Science at Queen Mary University of London, said: “Although Nim is a children’s game with full mathematical solutions, AlphaZero-style self-play can still create blind spots, making you competitive while missing the best moves in many positions.”

“This suggests that in future AI research, good performance alone is not proof that the system has learned the underlying principles. Reducing blind spots may require methods that capture abstract structure.”

—

“Impartial Games: A Challenge for Reinforcement Learning” by Dr. Bei Zhou, a researcher at Imperial College London, and Dr. Soren Rees-Leeder, a computer science professor at Queen Mary University of London, has been published in Machine Learning magazine.