Noah Smith: The Intelligence Explosion Debate Is the Wrong Frame for AI Safety Policy
Economist and Noahpinion author Noah Smith argues that whether or not an intelligence explosion occurs, the policy response must be the same: regulate the near-superhuman AI already arriving. Pacing the frontier cannot prevent an explosion if one is possible, and steady progress already produces economically equivalent outcomes, making the RSI debate a distraction from actionable governance.
Vernor Vinge predicted the Singularity would arrive between 2005 and 2030. Almost every AI safety discussion in San Francisco still centers on whether recursive self-improvement is near and whether we should deliberately slow AI progress to stop it from triggering an intelligence explosion. Noah Smith, the economist behind the Noahpinion newsletter, has grown increasingly skeptical that this is the right question to be asking.
Smith lays out the debate briefly before arguing it is misplaced. On one side, Ramez Naam, Theo Jaffee, and economists Luis Garicano and Alex Imas point to OpenAI data showing AI can still only automate short chunks of the research process, argue that diminishing returns mean each capability gain buys less speedup, and note there is no sign yet of acceleration in the overall rate of progress. On the other side, a METR report on Anthropic's internal research guesses that a majority (perhaps ≈80%) of the recent acceleration in AI progress is due to AI itself, and Dwarkesh Patel draws an analogy to mathematics — where models went from unable to add four-digit numbers in 2023 to solving a Millennium Prize problem in 88 hours — arguing that AI research has the same verifiable properties that made that jump possible.
Smith's central move is to argue that the answer to this debate does not change the policy picture. If an intelligence explosion is possible, it will happen, because no amount of pacing can prevent it. Halving the rate of AI progress would shift the threshold from, say, 2032 to 2037 — a few extra years that Smith calls 'small potatoes' for anyone who believes the explosion would be catastrophic. The only way to truly prevent it would be to halt AI progress completely, and Smith agrees with Vinge's 1993 argument that competitive advantage makes that politically impossible.