A British artificial intelligence startup has beaten scores of forecasting enthusiasts, including several professionals, in an international competition to predict events ranging from bust-ups between Donald Trump and Elon Musk to Kemi Badenoch being removed as Conservative party leader.
ManticAI, co-founded by a former Google DeepMind researcher, ranked eighth in the Metaculus Cup, run by a San Francisco-based forecasting company that provides predictions for investment funds and corporations. The contest required entrants to forecast the likelihood of 60 events over the summer, including which party would win the most seats in the Samoan general election and how many acres in the US would be burned by fires from January to August.
Ben Shindel, a professional forecaster who finished above Mantic but found himself behind AI at one point, said: “It’s certainly a weird feeling to be outdone by several bots at this point. We’ve really come a long way here compared with a year ago when the best bot was at something like rank 300.”
Toby Shevlane, co-founder of Mantic, described the performance as a milestone for using large language models in forecasting. “Some say LLMs just regurgitate their training data, but you can’t predict the future like that,” he said. “It requires genuine reasoning. The AI system often strongly disagreed with the community average, so AI forecasters could be an antidote to groupthink.”
Deger Turan, chief executive of Metaculus, estimated AI would be on a par with or surpass the best human forecasters by 2029, but noted that currently “human forecasters are doing better than AI forecasters” on complex tasks. Warren Hatch of Good Judgment, a firm co-founded by superforecasting expert Philip Tetlock, said: “The answer isn’t human or AI, but instead human and AI to get the best forecast possible as quickly as possible.”
Lubos Saloky, a human forecaster who placed third in the competition, echoed that sentiment: “I do not plan to retire. If you can’t beat them, merge with them.”
