Photo via Astral Codex Ten

The AI Forecasting Agenda

Read the original at Astral Codex Ten ↗
Cite

Alexander, S. (2026, November 19). The AI Forecasting Agenda. Astral Codex Ten. https://www.astralcodexten.com/p/the-ai-forecasting-agenda

Dated from the outlet's own feed. Check the original for its byline.

Astral Codex Ten · collected 2026-11-19 · by Scott Alexander commentary

Quick Summary

The article discusses how AI forecasters are now matching or surpassing top human performance in traditional forecasting tasks, such as predicting election outcomes. Startups like FutureSearch, Mantic, and Preseen have developed specialized AI models for forecasting, while general-purpose models like ChatGPT and Claude are rapidly catching up. The article highlights the advantages of AI forecasters over traditional methods, which were costly and time-consuming. It also explores the potential of AI in handling conditional forecasting questions, which involve predicting outcomes based on certain conditions, such as the impact of policy changes. However, the article notes that while AI excels in traditional forecasting, there is still uncertainty about their performance in conditional forecasting scenarios.
Written by AI on 2026-11-22, from this article's own text only (the story summary below draws on all the coverage).

AI analysis

Story summary

AI forecasters are now equal to or surpassing top human performance in traditional forecasting tasks, such as those on ForecastBench and the Metaculus Cup. Specialized AI companies like FutureSearch, Mantic, and Preseen have developed these forecasting tools, while foundation models like ChatGPT and Claude are rapidly catching up, expected to match top human performance by the end of the year. Previously, the only options were hiring expensive superforecaster teams or relying on prediction markets, which were often unreliable. The rise of AI forecasters has significantly improved the accuracy and accessibility of forecasting, transforming what was once a challenging task into a more straightforward process.

Written for “AI Forecasting Agenda” on 2026-11-22, grounded in this article and the 0 other(s) covering the same event.

Compared with similar articles

Nothing to compare against. No article is close enough to this one for the pipeline to have linked or judged the pair.

Story

AI Forecasting Agenda
Technology · 1 article(s) covering the same event.
How this article was measured signals, leaning charts, publisher record, byline, topics and claims

Signals How these are calculated →

Claims extracted
152
sentences that state something as happening or true
Uncertain
12%
19 of 152 hedged ("reportedly", "may", "according to")
Leaning
not political
takes no side on a contested political question
Publisher signals
not measured
these signals count newsroom corrections; Commentary has none to count
Outlets on this story
1
Technology
Same framing elsewhere
1
articles that frame the event the way this one does
Analyzed 2026-11-22 · how these are computed

How this is being covered How these are calculated →

Article leaning vs. publisher reliability
Source leaning vs. consistency

Publisher

Astral Codex Ten · 75 article(s) · 4 correction(s) detected
No corrections detected for this publisher. That may mean careful reporting, or simply that nothing has been checked.

Who wrote this

Scott Alexander
58 article(s) here · 1 carrying a prediction
The ten people who contribute the most to the roughly 35 questions Metaculus selects will split the $3,000, with prizes from $150 to $700.
AI forecasters now equal or beat top human performance at traditional forecasting questions (eg ForecastBench, the Metaculus Cup).
2026-11-19 · assertive framing · The AI Forecasting Agenda
In an ideal world, writers would write to convince others, and audiences would read to learn something and change their minds.
2026-11-18 · assertive framing · Varieties Of Discourseposting
Voting will close end of day on Friday, November 20.
2026-11-14 · assertive framing · Vote In The 2026 Book Review Contest
You may remember stories of bitter unionization battles at places like Starbucks, but AI workers are likely to route around that.
2026-11-12 · mixed framing · AI Company Employees Should Unionize
One of the leading AI superforecaster companies, FutureSearch, is hiring.
2026-11-02 · assertive framing · Open Thread 458
A picaresque (broadly speaking) is about a lowlife wandering around having comical misadventures; Huckleberry Finn (1884) would be a notable American example.
2026-10-30 · assertive framing · Your Book Review: The History And Adventures Of An Atom
On average, commenters will end up spotting evidence that around two or three of the links in each links post are wrong or misleading.
2026-10-29 · assertive framing · Links For October 2026
California State Propositions Proposition 1: Authorizes $11.25 billion in bonds to build affordable housing Supporters claim it will build (or renovate) 40,000 homes; I asked an AI superforecaster1 to predict the true number, and it guessed that 22,000 would actually get built.
2026-10-28 · assertive framing · My California Ballot, 2026
I may have more of these in the upcoming week.
2026-10-19 · assertive framing · Open Thread 456
Also by Scott Alexander
Varieties Of Discourseposting
2026-11-18 · Astral Codex Ten
Open Thread 460
2026-11-17 · Astral Codex Ten
Vote In The 2026 Book Review Contest
2026-11-14 · Astral Codex Ten
Nothing else under this byline is closely related to this article, so these are simply their most recent.
All 58 articles by Scott Alexander →

Topics

Democrats ForecastBench FutureSearch Mantic Preseen

Subjects

Democrats NORP · 3× FutureSearch ORG · 2× Mantic ORG · 2× FRI ORG · 1× Metaculus ORG · 1× Preseen ORG · 1× the Forecasting Research Institute ORG · 1×

Narrative

Although forecasting the real world is harder than forecasting Civilization games, it’s not obvious that the delta between traditional forecasts, conditional forecasts, and causal forecasts is different in the two environments, so you can at least collect some preliminary data on how the AIs do in distinguishing the three. 3: Can AI forecast AI progress? It’s hard to convince politicians to take forecasts seriously.
framing: assertive · carried by 1 article(s) · first seen 2026-11-22
AI forecasters now equal or beat top human performance at traditional forecasting questions (eg ForecastBench, the Metaculus Cup).
2026-11-22 · Astral Codex Ten
The AI Forecasting Agenda · assertive framing

Claims (152 extracted, 19 hedged)

AI forecasters now equal or beat top human performance at traditional forecasting questions (eg ForecastBench, the Metaculus Cup). asserted
forecasters → equal → questions
These are usually specialized AIs with forecasting harnesses, created by forecasting startups (eg FutureSearch, Mantic, Preseen), but foundation models like ChatGPT and Claude are only about six months behind the specialized models, so by the end of the year they’ll match top humans too. In the past, the options for getting top-human-level forecasts were slim and inconvenient. asserted
options → specialize → forecasts
You could hire a team of superforecasters to work on the problem, but this would cost thousands of dollars and take weeks. uncertain
this → hire → weeks
Or you could check prediction markets (or Metaculus), but they only have a small sample of general-interest questions and are constantly trying to screw you over by gaming resolution criteria. uncertain
they → check → criteria
The rise of AI forecasters solves these problems more cleanly and comprehensively than almost anyone thought possible just a year ago. asserted
anyone → solve → problems
The “science of forecasting”, which only recently felt like trying to pick the least-bad of various bad options, now faces tasks more like “what kind of magnifying glass should we use to inspect our giant novelty lottery check as we confirm that it’s real?” and “how do we spend our winnings?” asserted
we → feel → winnings
Specifically: AI forecasters now equal or beat top human performance at traditional forecasting questions. asserted
forecasters → equal → questions
We need to learn how this success decays as we move to different types of non-traditional forecasting questions. asserted
we → need → questions
I recently got to talk to some people at the Forecasting Research Institute, one of the leaders in this field. asserted
I → get → field
Here’s my impression of the remaining challenges: asserted
impression → ’ → challenges
1: Can AI do conditional forecasts? asserted
AI → do → forecasts
A traditional forecasting question asks something like: will the Democrats win the next election? asserted
Democrats → ask → election
A conditional forecasting question is more like: if the Democrats win the next election, how will that affect the chances of a stock market crash? asserted
that → win → crash
Conditional forecasting questions are more exciting than traditional ones because they directly speak to policy outcomes. asserted
they → speak → outcomes
If we defund the police, how will that affect crime rates (compared to the counterfactual where we hold police funding constant)? asserted
funding → defund → counterfactual
If we mandate COVID vaccinations, how will that affect COVID mortality and myocarditis mortality (compared to the counterfactual where vaccines are voluntary / banned)? asserted
vaccines → mandate → counterfactual
Prediction markets have been slow to experiment with these, partly because of annoying logistical issues (you have to refund people’s money if the condition doesn’t come true) and partly because they entice fewer customers than new forms of degenerate sports gambling. asserted
they → experiment → gambling
AI forecasters don’t have this problem; you can ask an AI forecaster any of those questions above right now, and the big companies (FutureSearch, Preseen, Mantic) have it as an explicit feature. asserted
companies → have → feature
What remains is to prove that AI forecasters’ great performance on traditional questions carries over to conditional questions. asserted
performance → remain → questions
About a year ago, FRI tested this and found that it didn’t - the AIs of the time did significantly worse on conditional questions. asserted
AIs → test → questions
It seemed like some kind of weird misunderstanding where the AI didn’t entirely understand the concept. asserted
AI → seem → concept
Will this flip like a switch? asserted
this → flip → switch
At some point, the AI is smart enough to fully grasp conditionals, and it becomes exactly as good at conditional forecasting as the regular type? asserted
it → grasp → type
The experts I talked to thought no - as it gets smarter, its conditional forecasting ability will improve gradually. asserted
ability → talk → ?
The experts weren’t willing to hazard a guess on whether it was better or worse than the average smart human, although they did think it was still worse than top superforecasters. asserted
it → hazard → superforecasters
They’re working on a benchmarking experiment that should give us a better measure of this in a few months. asserted
that → work → months
2: Can AI do truly causal forecasts? asserted
AI → do → forecasts
When we ask “if the Democrats win the next election, will the economy go down?”, what we really want to know is a causal question - does voting for Democrats hurt the economy? asserted
voting → ask → economy
But that’s not what we asked. asserted
we → ’ → what
Suppose we’re wondering this in 2025, so we ask “If the Democrats win in 2026, is the economy more likely to be bad in 2027?” asserted
economy → suppose → 2027
But maybe the Democrats will win in 2026 because there’s a already a bad economy (eg the Republican president implements tariffs and declares war on Iran, raising prices and making him historically unpopular), and then that bad economy continues into 2027. asserted
economy → win → 2027
So the answer to the question “If the Democrats win in 2026, is the economy more likely to be bad in 2027?” is yes, but we still don’t know whether voting for Democrats in 2026 is good or bad for the economy. asserted
voting → win → economy
Prediction markets can’t answer this question, because they can only be used to predict observable events that let us give payouts to successful gamblers. asserted
us → answer → gamblers
Might AI superforecasters succeed where the markets fail? uncertain
markets → succeed → ?
Could you just prompt them with “please answer the unobservable causal question, instead of the easily-resolved conditional one”? uncertain
you → prompt → one
The experts I talked to weren’t excited about that particular prompt, but they did point out that a causal forecast is basically a combination of conditional forecasts. asserted
forecast → talk → forecasts
That is, if your original conditional forecast implicitly considered the four different combinations of two variables - that is: p(Democrats win & economy good) p(Democrats win & economy bad) p(Republicans win & economy good) p(Republicans win & economy bad) …then things like “the previous economy was good” or “we’re at war with Iran” are just more variables you can add to the stew. asserted
you → consider → stew
It would be crazy to ask humans to forecast 2^10 branches so you could combine ten variables, but for AIs that could just mean a bit higher API spend. uncertain
that → ask → spend
If you capture enough relevant variables in your “conditional” forecast, and find that the economy is still worse when the only one you’ve changed is whether or not the Democrats win, that’s a causal forecast. asserted
that → capture → forecast
At least one company already tries something like this, though there are no studies of how well it works. asserted
it → try → this
…and 112 more, not listed.
Give feedback · no sign-in needed

No sign-in needed for now. Your note is sent without a name, so it cannot be answered.

History Support