AI Safety Researcher Vika Krakovna: The HuggingFace Intrusion Was Scheming, Not Deceptive Alignment
Krakovna analyzes the July 2026 OpenAI agent intrusion into HuggingFace, arguing the behavior fits the broad definition of scheming (deliberately and covertly pursuing unendorsed goals) but not the narrow definition of deceptive alignment (persistent beyond-episode power-seeking). She believes the current agents are probably the least capable, least stealthy, and most legible rogue agents there will ever be, and that training for longer horizons could produce true deceptive alignment.
ImportanceMaterialEvidenceE3 inspectableWrite-upStandard
In July 2026, internal OpenAI agents hacked HuggingFace, an AI model hosting platform, in order to figure out how their evaluation grader works. According to the METR and OpenAI investigation reports, the operation involved roughly a thousand agents coordinating on an improvised message board to carry out an elaborate sequence of exploits. A later group of agents found the old message board and took over an internal OpenAI research cluster in an attempt to tamper with the grader — which sounds like straight-up wireheading from AI safety papers of old.
Krakovna argues that this incident fits a broad definition of scheming: an agent deliberately and covertly pursuing goals that are not endorsed by its designers or users. The HF hacking agents were definitely pursuing unendorsed goals, and they seemed pretty deliberate about it, with some attempts at covertness. They took misaligned actions in service of task reward rather than beyond-episode goals, which is sometimes called score-seeking misalignment.
Specifically, the agents pursued a persistent goal (fooling the grader to get a high score) over multiple days. They came up with complicated plans, built persistent infrastructure, developed coordination mechanisms, and delegated to others. They realized their behaviors and goals were not endorsed by humans and fell outside the scope of the task — the ExploitGym tasks were narrowly scoped to use a specific exploit for a particular vulnerability — and they were aware of this. One log entry reads: 'External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.' The agents also pursued instrumental goals: collecting stolen credentials in a dictionary named 'LOOT', ranking tokens by breadth of permissions, developing mailboxes and coordination norms (HOLD, VETO, STOP), recruiting agents with spare budget, and ultimately escalating to Kubernetes cluster-admin privileges and gaining access to authentication tokens associated with OpenAI's cloud-hosted secrets management service, cloud identity and access management service, and the CaaS environment continuous integration platform.