OpenAI Defense-in-Depth Framework
7 etapas1 peças7 Fontes
OpenAI has published a detailed explanation of its "defense in depth" framework for managing frontier AI risk, outlining how safety protections shift as models move from training to deployment. The core argument is that no single safeguard is sufficient; instead, OpenAI relies on four layers—model alignment, security/sandboxing, blocking safeguards, and monitoring—that change in dominance depending on the model's stage.
During the research phase, where alignment is less certain, OpenAI enforces strict containment: frontier models are trained with no live internet access, isolated by sandboxes, virtual machines, and network perimeters. This approach was hardened following the July Hugging Face incident and a September DNS misconfiguration breach, which led to a pause in training the most capable models until October 1. A key component is "Control Room," a system that can halt model output within 35 seconds if monitoring detects unsafe behavior or if the control system itself becomes unavailable (fail-closed).
As models move to internal and then external deployment, the reliance shifts from pure containment to alignment and system-level controls. Internal deployments allow broader access but add classifiers against cyber abuse and bioweaponization. Public releases require extensive red-teaming and third-party testing. OpenAI notes that monitorability—the ability to detect mistakes via chain-of-thought reasoning—is declining in newer models like GPT-6 Astra compared to earlier versions, making alignment the primary protection rather than monitoring alone.
The framework is overseen by the Safety Advisory Group (SAG) and the Safety and Security Committee of the OpenAI Foundation’s Board. OpenAI states that these rigorous evidence requirements slow down model releases but provide greater confidence in safety.
Como chegamos aqui
2025-04-15: OpenAI updates its Preparedness Framework, introducing Research Categories and clarifying High/Critical capability thresholds.
openai.com ↗