Anthropic Letter to CA Governor Proposes Court Power to Block High-Risk AI Models

According to a report by INSIDE, artificial intelligence company Anthropic submitted a letter of opinion to California Governor Gavin Newsom on October 15, 2026. The letter formally responds to Executive Order N-9-26, signed by Newsom in September, which aims to establish regulatory frameworks for AI "kill switches" and safety oversight.

In the letter, Anthropic argues that when reviews determine an AI model poses severe risks, the government should be able to request courts to block its deployment. Specifically, Anthropic proposes legislation requiring large advanced AI developers to publish risk reports at least every six months, reviewed by Independent Verification Organizations (IVOs). If an IVO identifies risks and notifies the State Attorney General, and the AG determines the model poses a "risk of imminent catastrophic harm," they can petition a court for an order to suspend the model's deployment, internal use, or further development.

Anthropic stated that requiring both IVO assessment and a court ruling before suspension prevents abuse of power while allowing rapid response in emergencies. The company also expressed reservations about laws focusing solely on shutting down models, warning this could create a false sense of security since models executed in customers' isolated environments cannot be remotely withdrawn.

For comparison, OpenAI also wrote to Newsom on October 20, supporting amendments to California's SB 53 to require effective shutdown controls. However, OpenAI's letter did not propose a procedure for the government to seek court orders to pause models. These proposals remain part of the policy debate and have not yet been enacted into state law.