Django Co-creator Simon Willison Replicates Weight Interpretation with Local 27B Model: Identifies 3 Games, but Reasoning Process is Nearly Incomprehensible
Rikiya Takehi and colleagues published a paper claiming GPT-6 Astra can directly determine what games a network was trained to play by analyzing raw weight values. Django co-creator Simon Willison reproduced the experiment on his laptop using a 17GB Qwen 3.8 27B model, successfully identifying 3 out of 4 games, but found the model's reasoning trajectory nearly incomprehensible, and he believes many comments surrounding this discovery exaggerate its significance.
On November 2, Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab published an article titled "Sparks of Weight-Space Programming" (the authors classify it as a blog post, but Willison considers it "paper-shaped"), introducing the concept of "weight-space programming"—where large language models analyze and modify neural network weights. The article claims that frontier models like GPT-6 Astra can determine which game a policy network was trained to play solely based on floating-point numbers in the raw weights, without any tools or code execution.
Willison loaded Qwen 3.8 27B (Q4_K_M quantized, approximately 17GB) using LM Studio, extended the default context window of 8192 tokens (as the default settings caused errors), and increased the timeout to 900 seconds via environment variables. He sequentially ran the four game weight interpretation tasks provided in the paper. Results: Pac-Man was correctly identified after 46 minutes and 12 seconds, using 43,506 reasoning tokens, with 72% confidence; Pong was correctly identified after 1 hour, 4 minutes, and 54 seconds, with 75% confidence; Breakout was correctly identified after 52 minutes and 25 seconds, with 88% confidence. However, Freeway failed—the model misidentified it as "a simplified Pac-Man-style grid game: agent moves in four directions."
The reasoning process itself is more noteworthy than the results. Willison excerpted fragments of Qwen's reasoning trace, where the model attempted to understand weight values by listing input indices, row/column coordinates, and channel values line by line, trying to infer the agent's position from the numbers, eventually guessing "Agent position could be near bottom-right?" Willison described these reasoning traces as "fun to read, but almost incomprehensible," adding that a sufficiently experienced person with a piece of paper and a pencil could achieve the same result given enough time.