AI Safety Scholars Narayanan and Kapoor: Existential Risk Framing May Hinder Broad Societal Resilience

In an article published in their newsletter 'AI as Normal Technology', Princeton University computer science professors Arvind Narayanan and Sayash Kapoor discuss the strategic direction of the AI safety movement. They note that current debates typically revolve around two narratives: either AI existential risk is real and imminent, or claims to that effect are insincere hype or regulatory capture. The authors propose a third possibility: that these warnings are sincere but strategically wrong and counterproductive to AI safety.

The authors acknowledge that there is systematic underinvestment in defending against catastrophic and systemic risks. They cite pandemic preparedness as a primary example, noting that despite years of warnings and quantitative estimates, most recommendations were never implemented because voters reward relief spending rather than proactive prevention. Similarly, in cybersecurity, the lack of a culture modeling systemic risks means markets fail to appropriately price tail risks.

However, Narayanan and Kapoor argue that framing AI as an existential risk to mobilize public and policymakers may have negative consequences. First, doom-laden rhetoric is highly polarizing and can split along partisan lines, leading to policy uncertainty and reversals. Second, when the public realizes that predictions like 10% extinction risk lack robust quantitative models, they feel misled, which increases pushback against safety policies.