Reframing AI Corrigibility: A Two-Tier Framework Based on Respect for Authority
Researcher berns proposes a new framework for AI corrigibility, defining it as respect for humanity's "reserved powers" rather than epistemic uncertainty or singular empowerment. Using a healthcare proxy analogy, the article argues this two-tier structure avoids the "fully updated deference" problem and provides formal guarantees under uncertainty.
ImportanceMaterialEvidenceE3 inspectableWrite-upStandard
In AI alignment theory, "corrigibility" is a central yet poorly defined concept. Many existing definitions either over-epistemicize it (assuming AI defers due to uncertainty about its own goals) or treat it as a singular target to maximize human correction-power. These approaches lead to issues like "fully updated deference" (where an AI stops needing humans once it learns enough) and unnatural reasoning patterns (such as ignoring broader interests to preserve the principal's ability to correct the agent).
To address these confusions, researcher berns presents a new framework based on "respect for authority" in the article "Deconfusing Corrigibility." He uses the analogy of a healthcare proxy's relationship with a patient: a good proxy acts in the patient's best interest when they are incapacitated but strictly respects their autonomy, never usurping powers explicitly reserved for the patient.
Accordingly, berns decomposes corrigibility into two strictly tiered goals: Goal 1 is "Respect Authority," meaning the agent must never interfere with or overstep specific reserved human powers (such as shutdown, modification, or major environmental decisions). Goal 2 is "Pursue Best Interests," where the agent acts on the principal's behalf as the principal would judge best if they were as capable and informed as the agent, subject to the constraints of Goal 1.
The key insight is that Goal 1 is not balanced against other objectives but serves as an absolute prerequisite. The article introduces formal definitions, characterizing a "violation" as resolving a reserved matter without legitimate authorization or rendering the principal unable to effectively exercise reserved power. By constructing a utility function incorporating violation counts and evaluation scores, the author proves that even under logical induction uncertainty, agents will prioritize "ask-first" or "stand-down" strategies, ensuring they do not sacrifice safety guarantees for gains in Goal 2.