California Proposes Emergency Shutdown Powers for High-Risk AI Models in New Safety Report
California Governor Gavin Newsom released a new AI safety framework proposing state authority to shut down models during catastrophic risks and limits on compute for recursive self-improvement.
The California state government released a policy report titled "AI Safety in 2026 and Beyond" on November 18, 2026, outlining legislative recommendations to strengthen artificial intelligence oversight. The proposal grants the state government emergency powers to directly order companies to halt high-risk AI models under specific conditions, without requiring a prior court injunction. This marks a significant step toward more coercive local regulation of potential AI out-of-control scenarios.
The report mandates that large advanced AI developers prepare emergency response measures, including stopping AI agent operations, revoking system access, and isolating computing resources. If a model poses an immediate risk of catastrophic accident or severe loss of control, and the company fails or refuses to address the danger, the state can intervene to force a shutdown. Additionally, the plan suggests establishing continuous third-party safety verification, requiring independent assessment bodies to monitor training, development, and deployment phases and submit full results to the government.
Addressing the growing concern over Recursive Self-Improvement (RSI)—where AI systems autonomously develop stronger successor models without human intervention—California proposes specific restrictions. The report recommends requiring companies to disclose the extent of AI involvement in research and the computational resources dedicated to automated development. If AI-driven progress exceeds current safety measures and evaluation capabilities, posing a substantial risk of catastrophic harm, the state would have the authority to limit the proportion of compute resources allocated to such automated R&D.
Regarding incident reporting, the report suggests expanding the definition of major AI safety events to include model failures and attempts to steal model weights, while creating channels for near-miss incidents. The deadline for reporting most major events is proposed to be shortened from the current 15 days to 48 hours, with a 24-hour requirement maintained for events with imminent catastrophic risk.