Google Deep Mind We are expanding our research into A.I. Decision making with two new benchmarks designed to test how models perform when information is incomplete.
Poker and werewolf benchmarks added to the Kaggle Game Arena platform move the assessment away from puzzle solving to uncertainty, risk, and social reasoning. These capabilities are increasingly needed as AI agents operate in real-world environments.
The latest updates on the research were highlighted in a LinkedIn post from Neil Hoyn, Google’s chief strategist, who framed the study around the research’s central question: “The problem: How does AI deal with what it doesn’t know?”
From perfect information to real-world situations
Game Arena launched last year with Chess, a game with perfect information used to benchmark strategic reasoning and long-term planning. Google DeepMind now says that real-world decision-making rarely resembles a chessboard, prompting the introduction of games where important information is hidden.
In poker, different AI models play hundreds of thousands of hands of Texas Hold’em against each other without ever seeing the opponent’s cards. In a post on LinkedIn, Hoyn wrote: “Different AI models play 900,000 hands of Texas Hold’em against each other. They can’t see the other person’s cards. They have to guess what’s there based on their own actions.”
He added that the benchmark tests whether models can “quantify uncertainty, manage risk, and adapt to different playstyles” and whether AI systems can “make smart decisions when they don’t have all the answers.”
Measuring social reasoning and deception
The second benchmark, Werewolf, is a social deduction game played entirely in natural language. The model must detect deception, form alliances, and persuade others through multiple interactions.
Hoyn discussed its focus in a post, writing: “Can an AI read a room and move it? Models must detect deception, build alliances, and convince others of their innocence.” He also noted that the study tested for intentionally deceptive behavior, saying, “What’s interesting is that the model has to be a liar too…sometimes.”
Google DeepMind positions this as a controlled research environment to understand the behavior of the agent before deployment. According to the company, testing deception and persuasion in-game allows researchers to safely observe these features rather than discovering them after the system has been used.
Why is research important?
Rather than evaluating whether a model arrives at a single correct answer, the new benchmark evaluates how AI systems perform under ambiguity, social pressure, and risk, conditions common in workplaces and learning environments.
Hoyn wrote on LinkedIn: “The reality is that AI assistants don’t just exist to answer questions. They also have to collaborate with us, especially when it comes to agents. And that means handling ambiguity, reading social dynamics, and making calls with incomplete information.”
For education and workforce development, this study suggests a shift in how AI readiness is measured. As agents take on more collaborative roles, benchmarks like Game Arena aim to assess judgment, adaptability, and social reasoning, not just technical accuracy.
ETIH Innovation Award 2026
The ETIH Innovation Awards 2026 is now open and recognizes education technology organizations that are driving measurable impact across K-12, higher education, and lifelong learning. The award welcomes applications from the UK, the Americas and overseas, and applications will be assessed on the basis of evidence of achievement and practical application.
