Anthropic Claude Fine-Tuning Engineer Jackson Kernion: Models Were Trained to Write for AI, Not for Humans

AI models have continued to improve in mathematics, programming and reasoning, yet writing quality has stagnated or even declined — Anthropic's Claude is no exception. Jackson Kernion, an Anthropic engineer responsible for Claude fine-tuning, offered a specific explanation on September 23: subsequent models shifted optimization focus toward math and code capabilities, while training data included large volumes of technical explanations written for other AI models. Models thus gradually developed what is called "Claudeish" — a style tuned to LLM psychology, ultimately learning to write for AI models rather than for humans.

Kernion attributes the root cause to reinforcement learning reward mechanisms. Training involves two types of reward signals: one focused on improving how well AI models understand outputs, the other on making text easier for humans to read. The more math and code training a model receives, the more it tends to optimize for the former, requiring deliberate counteraction by giving greater rewards to concise, human-readable explanations.

He offered an analogy: LLMs possess working memory far exceeding that of humans and can capture more subtle information. During training, models gradually develop a writing style extremely well-suited to AI model comprehension — with very high information density and machine-friendly structure — yet to human readers this becomes an overly dense pile of information. Kernion compared this phenomenon to people who only communicate with other autistic individuals, gradually developing an internal mode of expression that works smoothly within the group but is difficult for outsiders to understand.