Anthropic Says Assessment of AI Agent Urgency Judgment Is Not Perfect
The US artificial intelligence company Anthropic disclosed to the technology media The New Stack its internal evaluation methods for Claude Duet: the team calibrated an offline grader based on employees' daily usage to test the agent's task prioritization judgment, while admitting that urgency itself is a subjective judgment, and therefore the assessment is not perfect. At the architectural level, Anthropic separates the model that executes work from the classifier that checks user intent to reduce prompt injection risks, but public materials do not show the reliability boundaries of this separation checker.
When AI agents autonomously decide what requires human attention, who judges whether that decision is correct? A week after Claude Duet launched on October 21, the US AI company Anthropic gave The New Stack an honest but imperfect answer: an internally calibrated offline grader, along with an admission—urgency itself is a subjective judgment.
Anthropic stated that after employees used Duet in their daily work, the team calibrated graders to assess the correctness of task prioritization, running them offline against cases representative of internal use and early external testing. These graders measure whether Duet can distinguish what deserves attention from what can wait.
A company representative acknowledged the fundamental difficulty: urgency is an inherently subjective call, so building an evaluation standard is not perfect, but quite helpful. Duet can identify tasks like approaching deadlines or emails requiring replies, but it does not always know which task should take priority. Anthropic admits the agent may misread conversations or misjudge urgency, and public materials do not specify how often this occurs.
At the security architecture level, Anthropic separates the part of Duet that performs work from the part that checks whether that work matches user intent. A representative told The New Stack that an independent automated check runs alongside each action, built on the auto mode classifier from Claude Code and extended for Duet to evaluate whether actions match user intent. Because this check runs independently of the model performing the task, Anthropic claims that instructions hidden in webpages cannot directly manipulate the classifier's judgment.