A UK-based AI startup, ManticAI, has made waves in the competitive world of forecasting by placing in the top ten of an international contest. Finishing eighth in the Metaculus Cup, the AI system proved more accurate than scores of human enthusiasts and even several professional forecasters. This achievement marks a significant milestone in the application of large language models for future-event prediction.
The contest required participants to assign probabilities to 60 different future events over a period of several months. The questions were designed to test analytical and predictive skills, covering topics from a potential public spat between Donald Trump and Elon Musk to the political fate of Kemi Badenoch within the Conservative party. The final scoring was based on the accuracy of these predictions on September 1st.
ManticAI’s strategy involves a novel approach to problem-solving. Instead of relying on a single model, it uses a team of different AI agents, including those from OpenAI, Google, and DeepSeek. Each agent is tasked with a specific role, such as historical research, scenario modeling, or current event analysis. This multi-faceted process allows the system to build a robust and well-reasoned forecast.
Toby Shevlane, ManticAI’s co-founder, highlighted that the system’s success refutes the idea that LLMs merely “regurgitate” training data. He argued that predicting the future requires “genuine reasoning.” Interestingly, he noted that the AI’s predictions often diverged from the human consensus, suggesting it could serve as a valuable tool to counteract groupthink and introduce more original perspectives into forecasting.
Despite this success, experts agree that humans currently maintain an advantage in areas requiring nuanced judgment and the ability to verify complex logical chains. However, the rapid progress is undeniable. As one human competitor who finished third, Lubos Saloky, wisely put it, the path forward may be to “merge with them” rather than compete, heralding a new era of human-AI collaboration in forecasting.