ReadingResearchRadarInvestment frameworkAI Company Primary Sources
Sign in / Sign up
← Back to reading
NewsMaterial3 min readDisconfirmAI Editorial

WIRED AI Reporter Puts AI Agents in Charge of a Premier League Fantasy Team: Division of Labor Improves Predictions, but AI Still Falls Short of Human Superforecasters

WIRED Senior AI Reporter Will Knight deployed a swarm of AI agents into the Premier League Fantasy Football game and found that assigning specific roles—such as reading news, analyzing match results, running models, and critiquing suggestions—to different agents improved prediction quality more effectively than simply scaling the number of agents to over one hundred. Haifeng Xu, a computer scientist at the University of Chicago, explained that AI still often lags behind human superforecasters because humans possess a deeper understanding of the real-world dynamics behind the news; however, competitions like the Metaculus Cup have shown this advantage is narrowing.

2026-10-29 02:30 GMT+8

Benching Erling Haaland—that was OpenAI’s Codex’s first recommendation for WIRED Senior AI Reporter Will Knight’s Premier League fantasy team. Knight unleashed a horde of AI agents into the Premier League Fantasy Football game, letting them draft their own teams, set lineups, and adjust rosters based on weekly news, aiming to answer a specific question: Can a group of robots predict better than humans?

This wasn’t Knight’s first attempt. A few years ago, he used a tool from the UK’s Turing Institute that modeled purely on historical data, completely unaware of player injuries or coaching changes, yielding recommendations that were astonishingly absurd. This time, he changed his approach: he fed Codex several relevant research papers first, then asked it to build a harness—a system of instructions and code that guides different agents through various steps—including checking recent news and running predictive models.

Codex quickly assembled an analysis framework that looked information-rich, but the initial recommendations remained foolish. Knight then instructed Codex to create a team of specialized agents: some responsible for reading news and blog posts, others for analyzing recent match results, more for running models, and a batch dedicated to critiquing suggestions. This change significantly improved the output. He subsequently increased the number of agents to over one hundred, hoping the system would become smarter, but aside from inflating his API bill, there was little change.

Sign in / Sign up
ReadingResearchRadarInvestment frameworkAI Company Primary Sources

Haifeng Xu, a computer scientist at the University of Chicago who studies the application of AI in market scenarios, believes that new models perform better in predictions fundamentally because they can digest and analyze massive amounts of real-world information and orchestrate more complex systems to generate guesses. Xu told Knight that AI is simply getting smarter at understanding the world, which is the root cause of their progress. Xu’s team began testing commercial models in prediction markets such as Kalshi and Polymarket, having models guess outcomes like the NBA Championship winner or whom President Trump would meet with. They quickly found that the models performed well, sometimes even surpassing average people without special guidance. In their tests, GPT-5 had the strongest predictive ability, followed by Grok 4, Claude Sonnet, Gemini, and Llama.

Xu’s team subsequently developed Prophet Arena, a benchmark for measuring model predictive capability. Another benchmark, Prediction Arena, developed by researchers at Harvard University and Arcada Labs, gave frontier models $10,000 to wager on prediction markets over 57 days. The models consistently lost money in the experiment, but researchers observed steady improvement over time. Xu told Knight that AI makes predictions most effectively using custom harnesses, and that having agents perform distinct tasks—reading news, collecting data, and analyzing the big picture—seems to be the most effective path.

However, AI still frequently lags behind highly skilled human experts known as superforecasters. Xu explained that although humans cannot consume vast amounts of information like AI, their understanding of the real world—the dynamics behind news stories—is deeper. Yet this advantage is shrinking. A competition called the Metaculus Cup recently showed that AI can sometimes defeat the best predictors. Another benchmark, ForecastBench, is expected to release results soon, potentially providing further evidence that AI can catch up to human levels. Knight believes that major labs will treat predicting the future as a new way to measure progress in machine intelligence.

For now, Knight’s AI team has not surpassed the league average, and he will not use any of its suggestions for his own team selection. But he expects AI to undoubtedly improve, perhaps soon dominating fantasy leagues with ease. He wrote that he somewhat hopes it won’t happen too quickly, as that would ruin the fun of earning bragging rights by winning the competition on one's own merit.

Sources

  • wired.comWIRED↗
Continue researching →