ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

AI Safety Blogger Zvi: Anthropic Doesn't Understand Why Its Draft Bill Was Unacceptable and Troubling

AI safety blogger Zvi Mowshowitz analyzes the response from Anthropic's federal affairs lead Anthony Cimino to the leaked US federal AI safety draft bill, arguing that the response addressed only peripheral details while missing the core problem: the bill's structure makes AI regulation harder rather than easier, and Anthropic's 'mistakes' systematically favor its own interests, contradicting the safety-organization identity its co-founders publicly articulated in 2023.

2026-10-21 07:10 GMT+8

Anthony Cimino, who leads Anthropic's federal affairs team, responded over the weekend to a leaked US federal AI safety draft bill that had been shared with the press, addressing four points of criticism one by one: the scope of federal preemption over state law, the evidentiary threshold for model blocking, risk trigger thresholds, and updates following summer incidents. Zvi Mowshowitz—a long-time AI safety blogger and author of Don't Worry About the Vase—argues that this response precisely reveals Anthropic's failure to understand what was centrally wrong.

Zvi's core judgment is that Cimino treats the draft's problems as fixable details, but the real issue is structural. The bill vests all power to address catastrophic and existential AI risks exclusively in the US federal government, with a preemption clause stronger than the existing FRONTIER Act, while simultaneously setting an extremely high threshold for federal authorities to block a model. In Zvi's words, 'you cannot have it both ways'—where preemption is needed (catastrophic risk), it demands federal monopoly power, but in areas where preemption could actually unify fragmented state laws (consumer protections, deepfakes), the bill proposes no preemption at all.

On model blocking, Cimino draws an analogy to emergency handling of occupational safety or chemical risks. Zvi rejects the analogy: occupational safety and chemical risks involve ordinary amounts of danger where harms can be weighed against benefits; frontier AI risks can involve mistakes that cannot be undone, including permanent loss of control over the future or very large catastrophic dangers. He argues that model blocking obviously needs a 'take it down first and we argue about it later' mechanism, and while the draft contains that layer of meaning (§13 allows the US Secretary of Commerce to petition a district court for a temporary restraining order), it is packaged behind a high evidentiary bar presented as a 'meaningful standard.'

Sign in / Sign up
ReadingResearchRadarInvestment framework

Zvi identifies a deeper structural contradiction: the draft makes the evaluator chosen by the lab itself likely the only window through which the US federal government can see inside the lab, while simultaneously setting the blocking threshold extremely high. This means the lab controls both the information channel and is nearly impossible to be forcibly stopped from outside. He acknowledges this can prevent abuse of power but concludes it looks like choosing to prioritize business interests.

The latter half of the article turns to a more fundamental question: if this is Anthropic's policy proposal at this moment, what even is Anthropic? Zvi cites Nathan Calvin's lengthy response, which lists Anthropic's accelerationist actions over the past year—announcing plans to build an in-house custom silicon chip team, signing compute deals exceeding $500 billion (according to The Information), increasing the percentage of AI development tasks that Claude 'leads' from 0% in March to 26% in August, switching to Responsible Scaling Policy V3 which abandons unilateral commitments to slow or stop development, and the comment 'you can't do safety from second place'—then asks: are these actions consistent with the role Anthropic's co-founders described in their 2023 essay 'Core views on AI safety,' namely to provide evidence that safety techniques may not prevent catastrophic risks and to sound the alarm?

Zvi argues that Anthropic's 'mistakes' share a common feature: they all lean toward the company's own interests. 'For every provision that was too weak or a giveaway to AI companies, why wasn't there a mistake pushing in the direction of being too strong?' He acknowledges that Anthropic has published important safety research, advanced third-party safety collaborations, and spoken forcefully about AI safety in many public forums, but notes that its speed of 'pushing the frontier' far exceeds its speed of 'pushing the safety frontier.'

Zvi's conclusion is not a demand that Anthropic support more aggressive legislation, but a demand for honesty: acknowledge that the draft was not the strongest bill on the table, acknowledge that its structure does not meet the moment, acknowledge that the combination of federal preemption with a high blocking threshold makes AI regulation harder rather than easier. He closes by quoting Anthropic co-founder Chris Olah: every frontier AI lab operates inside a set of incentives and constraints—commercial pressure, geopolitical pressure, and the older pressures of pride and ambition—that can sometimes conflict with doing the right thing. 'No matter how sincerely any of us intend to do the right thing, we will always be influenced by those incentives'—which is precisely why the existence of external critics matters enormously.

It should be noted that Zvi's position is an AI safety advocate's interpretation of frontier lab policy behavior, not an independently verified institutional analysis. The force of his argument lies in pattern recognition—the directionality that emerges when scattered actions and provisions are placed side by side—rather than authoritative interpretation of any single legal clause. Anthropic could reasonably respond that legislative drafting is an iterative process, that the strength or weakness of specific provisions reflects exploration of regulatory balance rather than a predetermined outcome. Zvi's rebuttal is: if the exploration is genuine, mistakes should be distributed in both directions.

Sources

  • lesswrong.com↗
Continue researching →