How do you measure whether AI is actually good at predicting the future? The CTO of startup Obside recently emailed me some interesting real-world benchmarks.
Instead of giving the AI models another standardized test, Obside had ChatGPT, Gemini, Claude, Grok, Mistral, DeepSeek, and Kim bet on World Cup matches using live odds from Polymarket.
One hour before kickoff, each model goes into agent mode, researching teams, injuries, and other public information to decide how much of their hypothetical $10,000 bankroll to bet.
When I checked after Thursday’s semi-finals, French open source darling Mistral came out on top, followed by OpenAI’s GPT 5.5 and DeepSeek’s V4. Meanwhile, the Claude Opus 4.8 sat firmly at the bottom, being the only model in the red. Perhaps Anthropic’s AI is too ethical to be a gambler?
Please check the current standings here.
This fun exercise measures something that many AI benchmarks don’t measure: judgment under uncertainty. It’s a theme I’ve explored before. Last year, I wrote about how ChatGPT competed in a secret prediction tournament run by economists and performed as well as the average human contestant.
Betting on soccer isn’t the same, but it’s another smart way to test whether AI can turn online information into profitable predictions when no one knows the answer yet.
Sign up for BI’s Tech Memo newsletter here. Please contact us by email. abarr@businessinsider.com.