AI Commentator Zvi Mowshowitz: Goodfire's 'Neural Networks Are Already Clean' Is a Major Update to Interpretability Methodology, But Far From Solving Alignment
Zvi Mowshowitz dissects Goodfire researcher Nick Cammarata's claim—that roughly 90% of neurons in large language models have a single core concept rather than arbitrary superposition—and highlights its strongest counterexample: a Colorado neuron presented as a 'single-concept' exemplar that fires equally strongly on Tornado framework code. Mowshowitz considers this a substantial update to neuron interpretability methodology, but Eliezer Yudkowsky's 'screwdriver' analogy and Neel Nanda's objections about weak activations indicate that enumerating neuron meanings remains far from predicting model behavior.
ImportanceMaterialEvidenceE3 inspectableWrite-upStandard
Goodfire researcher Nick Cammarata published a long essay whose central claim is that in modern large language models, roughly 90% of neurons have one main job with small related offshoots; about 7% have two related jobs; and only 2–4% are not yet understood. This directly contradicts the previously dominant 'superposition hypothesis'—the idea that models pack unrelated concepts into the same neurons. Cammarata argues that what was interpreted as arbitrary packing is mostly a pattern he calls 'recursive generalization': neurons grow offshoots the way tree branches sprout sub-branches, rather than randomly stacking unrelated concepts.
AI commentator Zvi Mowshowitz published a layered analysis of this claim on October 19, 2026. He acknowledges it as 'an important update about how LLMs and minds form and operate,' while explicitly stating that it 'does not tell us that interpretability will soon be solved, or is about to be made into a science, let alone that alignment will soon be handled.' Mowshowitz positions himself 'somewhere in the middle': great work, assuming the work checks out.
The most heated part of the debate is not the claim itself but its interpretation. OpenAI researcher Adam Majmudar compared it to the 'Attention Is All You Need' paper, saying it moves the field from 'is alignment even solvable' to 'we may solve alignment in the next year.' Mowshowitz reacted strongly, calling this precisely the mistake Eliezer Yudkowsky would rant about—'Yes, this is progress. No, this is not an MRI for AI. That is still a long way off.'