LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · AI存在性风险之争Estructural2 min de lecturaEditorial de DisconfirmAI

AI Safety Scholars Narayanan and Kapoor: Existential Risk Framing May Hinder Broad Societal Resilience

Princeton professors Arvind Narayanan and Sayash Kapoor argue in their newsletter 'AI as Normal Technology' that while society is indeed underinvested in catastrophic risk defense, overemphasizing existential risk may lead to political polarization and resource misdirection. They advocate for a 'big tent' AI safety movement focused on specific threats like cyber and bio-risks, rather than solely targeting bans on superintelligence.

2026-10-02 00:34 GMT+8

In an article published in their newsletter 'AI as Normal Technology', Princeton University computer science professors Arvind Narayanan and Sayash Kapoor discuss the strategic direction of the AI safety movement. They note that current debates typically revolve around two narratives: either AI existential risk is real and imminent, or claims to that effect are insincere hype or regulatory capture. The authors propose a third possibility: that these warnings are sincere but strategically wrong and counterproductive to AI safety.

The authors acknowledge that there is systematic underinvestment in defending against catastrophic and systemic risks. They cite pandemic preparedness as a primary example, noting that despite years of warnings and quantitative estimates, most recommendations were never implemented because voters reward relief spending rather than proactive prevention. Similarly, in cybersecurity, the lack of a culture modeling systemic risks means markets fail to appropriately price tail risks.

However, Narayanan and Kapoor argue that framing AI as an existential risk to mobilize public and policymakers may have negative consequences. First, doom-laden rhetoric is highly polarizing and can split along partisan lines, leading to policy uncertainty and reversals. Second, when the public realizes that predictions like 10% extinction risk lack robust quantitative models, they feel misled, which increases pushback against safety policies.

Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

More importantly, the existential risk framework may lead to misdirection. If the threat is defined broadly as existential, the only apparent policy response is banning the development of superintelligence. Such bans are definitionally ambiguous and do nothing to address catastrophic and systemic risks from existing AI systems. In contrast, defenses against cyber attacks, amplified bio-risks, and accumulative institutional decay require disparate, long-term resilience building rather than a single mammoth ban.

Therefore, the authors call for a 'big tent' AI safety movement. This movement should include members who are not concerned about x-risk but care about specific catastrophic risks, adopt pluralist values rather than a single utilitarian framework, and focus on low-hanging fruit in policy such as transparency, liability, and embedded evaluation. They argue that policy-by-panic often leads to blunt and ineffective solutions, whereas a broad coalition based on shared interests and concrete threats is more likely to achieve effective long-term governance.

Fuentes

  • normaltech.ai↗
Tema

AI存在性风险之争

围绕人工智能(AI)是否构成人类“存在性风险”(Existential Risk, x-risk)的争论已超越技术范畴,演变为关于资源分配、监管优先级及社会韧性的政治与伦理博弈。以普林斯顿大学Arvind Narayanan和Sayash …

5 pasos1 piezasActualizado 2026-10-23 12:03

Cómo llegó hasta aquí

  1. 2024-07-26普林斯顿大学教授Arvind Narayanan和Sayash Kapoor发表文章,批评AI存在性风险的概率估算缺乏科学依据,认为其过于投机而无法指导公共政策。normaltech.ai ↗
  2. 2025-06-24图灵奖得主、清华大学人工智能学院院长姚期智在论坛上指出,大语言模型的欺骗行为可能引发“生存性”风险,包括生物风险和大模型失控,呼吁建立紧急评估系统。tsinghua.edu.cn ↗
  3. 2026-09-23布鲁金斯学会发布多位学者评论,探讨AI存在性威胁的真实性及监管可行性,提及前Anthropic员工Jacob Coxon离职警告及OpenAI牵头成立标准组织等事件。brookings.edu ↗

Lo que aún está pendiente

  • 如何量化比较“超级智能失控”与“现有网络安全/生物风险”的资源投入优先级?
  • 美国参议院桑德斯提出的《禁止人工超级智能法案》及纽约市议会的“终止开关”法案最终立法结果如何?
  • 国际间是否会就AI能力红线达成具有约束力的“条件性安全条约”?
Leer el tema completo →
Continuar investigando →