LectureRechercheRadarCadre d'investissement
Connexion / Inscription
← Retour à la lecture
ActualitésMatériel2 min de lectureRédaction DisconfirmAI

OpenAI Details Anti-Distillation Defenses: Attack Success Rate for Reasoning Extraction Drops to 0.2%

OpenAI reports that new safeguards in GPT-6.2 Astra reduced the attack success rate for extracting protected reasoning from 54% to 0.2%, highlighting the security risks of adversarial distillation.

2026-11-20 18:00 GMT+8

OpenAI has published a detailed account of its defenses against adversarial distillation, a technique where bad actors use a frontier model's outputs to train another model with similar capabilities but without appropriate safeguards. The company emphasizes that this poses significant safety risks, particularly in dual-use domains like cybersecurity and biology, where a distilled model could assist in attacks that the original model would have refused.

A core component of OpenAI's strategy is protecting the model's "chain-of-thought" or intermediate reasoning steps, which are valuable training data for student models. In an internal adaptive evaluation using automated red-teaming, OpenAI tested whether attackers could recover this protected reasoning. The results showed a dramatic improvement in GPT-6.2 Astra: the average attack success rate fell from 54.0% in the previous version (GPT-6 Astra) to just 0.2% in GPT-6.2 Astra after implementing updated safeguards.

The company also shared findings from an internal study on how cyber capabilities transfer through distillation. When students were trained on exact reasoning traces rather than just final responses or summaries, they generally outperformed those trained on summaries, especially at larger inference budgets. This suggests that protecting exact reasoning traces can limit the transfer of risky cyber capabilities, even when the underlying tasks are benign.

OpenAI's defense stack combines multiple layers: defensive training to help models recognize and refuse extraction attempts, input/output classifiers to detect leaks, and API-level protections that tie protected reasoning to specific conversation histories. If a message is rewritten, the subsequent reasoning is dropped. Additionally, older model families with weaker safeguards are prevented from reading the protected reasoning of newer models like GPT-6.2 Astra and Sol.

Connexion / Inscription
LectureRechercheRadarCadre d'investissement

Beyond technical measures, OpenAI reported active enforcement against coordinated distillation campaigns. Weekly actions have targeted thousands to tens of thousands of accounts, with rates increasing significantly this summer. The company previously attributed a core part of a disrupted campaign to individuals associated with Moonshot AI, developer of Kimi, underscoring the national security implications of unauthorized capability extraction.

Sources

  • openai.comOpenAI↗
Continuer la recherche →