ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

Noah Smith: The Intelligence Explosion Debate Is the Wrong Frame for AI Safety Policy

Economist and Noahpinion author Noah Smith argues that whether or not an intelligence explosion occurs, the policy response must be the same: regulate the near-superhuman AI already arriving. Pacing the frontier cannot prevent an explosion if one is possible, and steady progress already produces economically equivalent outcomes, making the RSI debate a distraction from actionable governance.

2026-11-20 18:13 GMT+8

Vernor Vinge predicted the Singularity would arrive between 2005 and 2030. Almost every AI safety discussion in San Francisco still centers on whether recursive self-improvement is near and whether we should deliberately slow AI progress to stop it from triggering an intelligence explosion. Noah Smith, the economist behind the Noahpinion newsletter, has grown increasingly skeptical that this is the right question to be asking.

Smith lays out the debate briefly before arguing it is misplaced. On one side, Ramez Naam, Theo Jaffee, and economists Luis Garicano and Alex Imas point to OpenAI data showing AI can still only automate short chunks of the research process, argue that diminishing returns mean each capability gain buys less speedup, and note there is no sign yet of acceleration in the overall rate of progress. On the other side, a METR report on Anthropic's internal research guesses that a majority (perhaps ≈80%) of the recent acceleration in AI progress is due to AI itself, and Dwarkesh Patel draws an analogy to mathematics — where models went from unable to add four-digit numbers in 2023 to solving a Millennium Prize problem in 88 hours — arguing that AI research has the same verifiable properties that made that jump possible.

Smith's central move is to argue that the answer to this debate does not change the policy picture. If an intelligence explosion is possible, it will happen, because no amount of pacing can prevent it. Halving the rate of AI progress would shift the threshold from, say, 2032 to 2037 — a few extra years that Smith calls 'small potatoes' for anyone who believes the explosion would be catastrophic. The only way to truly prevent it would be to halt AI progress completely, and Smith agrees with Vinge's 1993 argument that competitive advantage makes that politically impossible.

Sign in / Sign up
ReadingResearchRadarInvestment framework

Nor does Smith believe alignment research can make an explosion safe. Pacing the frontier so that alignment work can catch up might help in a world of steady linear progress, but in a world of sudden superexponential gains, alignment research is immediately left behind. Even if there is a Singularity for understanding alignment, it would be the superintelligent AI itself that carries it out and decides its own alignment.

The second half of Smith's argument is that the explosion is not distinguishable from what is already happening. AI going from unable to add two numbers to solving problems too hard for all human mathematicians combined in three years is not counted as an intelligence explosion. At the current rate of steady progress, Smith writes, AI will become superhuman in most domains in a fairly small number of years — even robotics and physical tasks look conquerable with the type of AI that powers Claude and GPT. This means AI will basically look like a god within our lifetimes whether or not it looks like the infinitely self-improving gods of Vinge's novels.

Smith then turns to economics. He notes that technologies usually reach a threshold of usefulness where adoption makes economic sense, creating rapid but localized productivity gains that finish quickly. Fiszbein et al. (2024) shows this for electrification; Bartel (2005) shows it for IT-enabled machines. Each invention creates many small local Singularities, and only when you add them up across the economy do you get something that looks like a big smooth wave. A suddenly superintelligent AI is not limited by locality — it could by definition take over all or most human tasks if it wanted to. But if no explosion occurs, those takeovers are likely to be staggered and irregular, and the economic effect after the first few years probably will not be that different.

The policy conclusion follows directly: we need to accept that nobody is pacing the frontier enough to prevent the emergence of superintelligence, and that nobody is likely to do so. The urgent work is regulating the AI we already have — competition policy to ensure benefits are not monopolized by one company, preventing AI-enabled bioterror, making sure economic gains are widely shared. Smith's framing is deliberately provocative: if the machine god awakes, then it awakes, and we will simply deal with it. In the meantime, he argues, we need to focus on dealing with the AI we have now.

The strongest objection to Smith's argument is that he may be underestimating the value of those few extra years. If alignment research is genuinely close to a breakthrough — a result that would make superintelligent AI safe regardless of how quickly it arrives — then pacing is not small potatoes but the difference between catastrophe and survival. Smith acknowledges this possibility but dismisses it, arguing that in a superexponential world, alignment research cannot keep pace regardless. Whether that dismissal is justified depends on an empirical question about the difficulty of alignment that Smith does not resolve.

Sources

  • noahpinion.blog↗
Continue researching →