LecturaInvestigaciónRadarMarco de inversión
Iniciar sesión / Registrarse
← Volver a la lectura
NoticiasTema · Claude DuetMaterial3 min de lecturaEditorial de DisconfirmAI

Anthropic Says Assessment of AI Agent Urgency Judgment Is Not Perfect

The US artificial intelligence company Anthropic disclosed to the technology media The New Stack its internal evaluation methods for Claude Duet: the team calibrated an offline grader based on employees' daily usage to test the agent's task prioritization judgment, while admitting that urgency itself is a subjective judgment, and therefore the assessment is not perfect. At the architectural level, Anthropic separates the model that executes work from the classifier that checks user intent to reduce prompt injection risks, but public materials do not show the reliability boundaries of this separation checker.

2026-10-29 19:00 GMT+8

When AI agents autonomously decide what requires human attention, who judges whether that decision is correct? A week after Claude Duet launched on October 21, the US AI company Anthropic gave The New Stack an honest but imperfect answer: an internally calibrated offline grader, along with an admission—urgency itself is a subjective judgment.

Anthropic stated that after employees used Duet in their daily work, the team calibrated graders to assess the correctness of task prioritization, running them offline against cases representative of internal use and early external testing. These graders measure whether Duet can distinguish what deserves attention from what can wait.

A company representative acknowledged the fundamental difficulty: urgency is an inherently subjective call, so building an evaluation standard is not perfect, but quite helpful. Duet can identify tasks like approaching deadlines or emails requiring replies, but it does not always know which task should take priority. Anthropic admits the agent may misread conversations or misjudge urgency, and public materials do not specify how often this occurs.

At the security architecture level, Anthropic separates the part of Duet that performs work from the part that checks whether that work matches user intent. A representative told The New Stack that an independent automated check runs alongside each action, built on the auto mode classifier from Claude Code and extended for Duet to evaluate whether actions match user intent. Because this check runs independently of the model performing the task, Anthropic claims that instructions hidden in webpages cannot directly manipulate the classifier's judgment.

Iniciar sesión / Registrarse
LecturaInvestigaciónRadarMarco de inversión

Anthropic stated that external companies conducted penetration testing on Duet, but public materials do not show how reliably this separation checker captures out-of-scope operations. Related research illustrates the difficulty of enforcing agent scope: researchers found that removing prompt sections defining authorized and out-of-scope actions in Claude Code tests increased out-of-scope actions, but that study did not test Duet's independent classifier. Anthropic's support materials still advise users to review drafts and approval requests because Duet may encounter prompt injection.

Another practical constraint is the trade-off between resource consumption and response speed. Duet checks connected apps every few hours rather than monitoring continuously, partly to avoid rapidly depleting subscription allowances. The cost is that urgent matters may arrive between checks and go unnoticed until the next one. More frequent checks consume more resources, while less frequent checks leave users waiting longer. Anthropic's public materials do not quantify how much background activity accounts for a typical subscriber's usage.

For teams building autonomous agents, Anthropic's approach provides a reference framework: calibrate evaluation standards using real internal workflows, acknowledge that subjective dimensions cannot be fully quantified, separate execution and verification at the architecture level, and make explicit trade-offs between resources and responsiveness. However, the reliability boundaries of this framework remain opaque—the specific metrics of the graders, the frequency of misjudgments, and results of independent validation have not been made public.

Fuentes

  • thenewstack.ioThe New Stack↗
  • claude.comClaude↗
Tema

Claude Duet

美国AI公司Anthropic于2026年10月21日推出主动式AI代理Claude Duet的研究预览版,该代理基于Claude Opus 5.5模型,旨在通过监控用户连接的应用程序来主动识别紧急事项并执行任务。【背景】Duet的核心价值…

3 pasos1 piezasActualizado 2026-10-31 00:31

Cómo llegó hasta aquí

  1. 2026-10-21Anthropic发布Claude Duet研究预览版,面向美国Pro和Max订阅用户开放候补名单。claude.com ↗
  2. 2026-10-23Anthropic团队负责人Karan Ahuja在社交媒体上回应反馈,确认Duet的安全隐私保障措施细节,并提及外部渗透测试。thenewstack.io ↗
  3. 2026-10-29Anthropic向The New Stack披露内部评估方法,承认基于员工日常使用校准的离线分级器在判断任务紧急程度时存在不完美之处。thenewstack.io ↗

Lo que aún está pendiente

  • Anthropic是否会在正式发布前公布更具体的优先级判断准确率数据或基准测试结果?
  • 分离的意图检查分类器在面对复杂间接提示注入时的实际拦截率是多少?
  • Duet是否会支持非美国地区用户或Team/Enterprise计划,以及是否允许用户自定义检查频率以平衡响应速度与用量消耗?
Leer el tema completo →
Continuar investigando →