spaCy Author Honnibal Critiques MIT Astra Study: Conflating 'Reading Weights' with 'Modifying Weights' is Misleading
Matthew Honnibal, co-founder of Explosion and author of spaCy, published an article arguing that the recently viral MIT Astra study is seriously misleading. While the study claims the model can understand and modify other models just by observing raw weights, Honnibal's analysis reveals that the experiments involving large language models like Qwen actually rely on Python code execution and multiple forward passes, which are fundamentally different from the 'pure weight reading' conditions used for small models. He acknowledges the progress in AI's ability to act as an 'engineer' but criticizes packaging these distinct technical paths as a single 'superpower,' noting it fuels industry hype.
Recently, a study by MIT researchers regarding the Astra model caused a sensation in the AI community. The research claimed that Astra could identify tasks performed by small neural networks simply by observing raw weight numbers, and even modify the weights of the Qwen3.5-0.8B model without using gradients, boosting its math score from 7.9% to 48.1%. Many netizens interpreted this as 'AI starting to read another AI's brain,' implying a unified, deep capability for model comprehension.
However, Matthew Honnibal, author of spaCy and co-founder of AI company Explosion, publicly questioned the presentation of this research. He argued that the study's experimental design resembles a stage magic 'misdirection': magicians may use different techniques to perform the same trick, but audiences often assume there is only one mysterious method. Honnibal pointed out that 'reading weights' and 'modifying weights' in the Astra study are actually two completely different technical paths, and bundling them together easily leads to reader misunderstanding.
After carefully reviewing eight sets of experiments in the paper, Honnibal found that the experiments strictly adhering to the condition of 'observing raw weights only' were limited to networks with very small parameter sizes (between 17K and 172K). In these experiments, Astra indeed did not run Python code or execute the target network, but instead placed the weight floating-point numbers directly into the context for analysis. While this demonstrates impressive pattern recognition, it remains far from truly understanding modern large language models.