読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニューストピック · AI代理可控自主权机制重大2 分で読めるDisconfirmAI 編集部

Two Sigma Architect John Feminella: Five Composable Controls for AI Agent Autonomy Illustrated by Citibank's $900M Error

John Feminella, who works on AI architecture and engineering at Two Sigma, uses the 2020 incident where Citibank accidentally sent about $900 million to Revlon’s lenders instead of an $8 million interest payment to discuss granting AI agents real autonomy without locking everything down. He outlines five composable controls: an agent registry, risk-based review, tool access that follows an agent’s declared purpose, shared firm-wide evals, and classifiers that send humans only the approvals that matter. Views are his own, not Two Sigma’s.

2026-10-27 23:00 GMT+8

In 2020, Citibank accidentally sent about $900 million to Revlon’s lenders instead of an $8 million interest payment, even though the procedure, the warning pop-up, and three approvers all did what they were supposed to. This famous operational failure highlights the risks inherent in traditional human-in-the-loop processes.

John Feminella, who works on AI architecture and engineering at Two Sigma, uses that story to show how to give AI agents real autonomy without locking everything down. In his recent video episode, he argues that simple restriction is not the solution; instead, sophisticated control layers are required.

Feminella walks through five composable controls designed to address governance gaps in AI agent deployment:

  1. An agent registry: Clearly identifying which agents exist within the system.
  2. Risk-based review: Adjusting scrutiny levels based on the potential impact of actions.
  3. Tool access that follows an agent’s declared purpose: Ensuring agents can only use tools relevant to their stated tasks.
  4. Shared firm-wide evals: Establishing unified testing standards to validate agent behavior across the organization.
ログイン / 新規登録
読書リサーチレーダー投資フレームワーク
  • Classifiers that send humans only the approvals that matter: Filtering routine requests to mitigate "approval fatigue" by escalating only critical decisions.
  • The episode was published on October 27, 2026, by the AI Engineer channel. Feminella explicitly notes that these views are his own and do not represent Two Sigma's official position. For engineers and managers exploring enterprise AI agent integration, this framework offers concrete architectural reference points.

    ソース

    • youtube.com↗
    トピック

    AI代理可控自主权机制

    随着大语言模型代理(Agentic AI)从对话转向执行代码和调用外部工具,如何赋予其真正的自主权同时确保人类控制成为核心议题。2026年10月,Two Sigma架构师John Feminella提出五项可组合的控制措施,包括代理注册表和…

    6 ステップ1 件更新日 2026-10-28 22:01

    経緯

    1. 2025-10-30Stanford大学William Overman等人发表《The Oversight Game》,提出通过最小控制接口让代理学习何时自主行动、何时请求人类监督,以实现内在对齐。arxiv.org ↗
    2. 2025-12-02IBM软件团队提交预印本《AGENTSAFE》,提出统一治理框架,将MIT AI风险库操作化为设计、运行时和审计控制,强调加密追踪和动态授权。arxiv.org ↗
    3. 2026-04-15Philipp Hacker和Matthias Holweg发布工作论文,分析欧盟AI法案下代理的法律地位,建议实施基于风险的“红绿灯”任务授权系统和不可委托法律行为清单。arxiv.org ↗

    未解決の問題

    • 现有护栏机制在多轮交互和复杂工具链中是否能提供持久且可执行的运行时控制?
    • 如何在法律框架内界定AI代理自主决策的责任归属,特别是在欧盟合同法背景下?
    • 自动化监督的可扩展性是否会削弱人类监督的合法性和有效性?
    トピック全文を読む →
    リサーチを続ける →