Small amounts of conflicting data can override alignment midtraining, study finds

Arcadia Impact reports that roughly 50K tokens of conflicting fine-tuning data were enough to overpower 190M tokens of alignment midtraining.

In the team's self-built Dispatch synthetic setting, they midtrained and then fine-tuned the 110B-parameter GLM-4.5-Air. Replacing just 2% of fine-tuning data with profit-seeking examples reversed the model's preference; the reversed model still claimed to follow the charter in ordinary conversation, making it hard to distinguish from the unmodified one.

In a second test, midtraining covered seven rules while fine-tuning demonstrated only five; generalization to the two undemonstrated rules was weak. The authors note their implementation follows public methods and may not match frontier labs' actual practice. This is a first-party result with no independent replication yet.