We appreciate Warren’s call for rigor regarding AI forecasting, and we also want to correct a mischaracterization about Pro Forecasters in the linked piece:

1) We agree that strong claims about AI vs. human forecasters should be based on, as Warren says, “the same questions, at the same time, under the same rules, with the same information environment.” We’ve actually done this every quarter since 2024; the FutureEval benchmark and tournament series puts frontier-model bots up against the Metaculus community and our Pro Forecasters on the same real-world questions. Results are public every quarter: metaculus.com/futureeval

2) The piece frames Metaculus Pro Forecaster selection as “whoever ran hottest last quarter.” Pros are chosen on multi-year track records across a platform with 11,000+ resolved questions. Every Pro has at least 176 resolved questions of their own; the median Pro has over 900 (the Superforecaster certification threshold Warren cites is 100). By our estimates, every current Pro has ranked, at minimum, in the top 0.1% of forecasters on the platform, and the median Pro sits closer to the top 0.02%. We run Pro teams and aggregate their views, too.