The Artificial Intelligence System has beaten many prediction enthusiasts, including several experts, in a contest to predict events that have been removed from Conservative leadership from bust-ups between Donald Trump and Elon Musk to bust-ups between Kemi Badenok.
The UK AI startup, co-founded by former Google Deepmind researchers, is ranked in the top 10 international forecasting contests, with participants having to predict the likelihood of 60 events in the summer.
Manticai ranked 8th in the Metaculus Cup and is run by a San Francisco-based forecasting company that seeks to predict the future of investment funds and companies.
AI performance is still behind the best human predictors, but some believe that AI could surpass humans sooner than expected.
“It certainly feels strange to lose to a few bots at this point,” said Ben Sindel, one of the professional predictors who found themselves behind the AI during the contest before finishing on the mantic. “We've come a really long way here compared to a year ago when the best bots were like Rank 300.”
The Metacruz Cup questions included which party would win the most seats in the Samoan general election, and whether the fires from January to August would burn a few acres of the US. The contestants were scored on how much they predicted the outcome as of September 1st.
“What Munch did is impressive,” said Degar Turan, Metacruz's CEO.
Turan estimated that AI would be face value or better than the best human predictors by 2029, but generally stated that “human predictors are doing better than AI predictors now.”
In complex predictions that rely on predictions of interrelated events, he said, AI systems can struggle to perform logic validation checks when converting knowledge into final predictions.
Mantic breaks down prediction problems into different jobs and assigns them to a roster of machine learning models such as Openai, Google, and Deepseek, depending on their strengths.
Mantic co-founder Toby Shevlane said its performance is a milestone for the AI community, using large language models for prediction.
“Some people say that LLMS only regurgizes training data, but we can't predict such a future,” he said. “We need authentic inference. We can say that our system's predictions are more original than most human entrants, as people often gather average community predictions. AI systems often opposed them.
Mantic's systems deploy a variety of AI agents to assess what is happening now, perform historical research, game out the scenario, and then predict what will happen next. The strength of AI prediction is the ability to work hard and lasts a long time and is essential for effective prediction.
They can easily work on dozens of complex problems at a time and revisit each day to learn from information changes. Human predictions also use intuition, but Sindel is one of the human predictors who think this could emerge in AI.
“Intuition is very important, but I don't think it's human in nature,” he said.
Top-class human super forayers say they are the best. Philip Tetlock, co-author of the bestselling book SuperForeCasting, published a study this year that found that experts average top-performing bots.
Turan said that in complex predictions that rely on predictions of interrelated events, AI systems can find logical inconsistencies in the output and struggle to perform validation checks.
“We've seen a lot of effort and we've seen a lot of money,” said Warren Hatch, CEO of Good Judgement, a forecasting company co-founded by Tetlock. “We expect AI to be good at questions in certain categories, such as monthly inflation.
Or, as Lubos Saloky, the human forecaster who came in third in the Metaculus Cup, said, “I'm not going to retire. If you can't beat them, I'll merge with them.”
