LectureRechercheRadarCadre d'investissement
Connexion / Inscription
← Retour à la lecture
ActualitésSujet · 智能体错误数据集Prise rapide1 min de lectureRédaction DisconfirmAI

50,000 agent error pairs released, proposed fixes lift pass rate to 51.1%

Authors' own tests show proposed corrections raise replay verifier pass rates from 18.4% to 51.1%; failure-trace reuse is worth tracking.

2026-10-01 12:00 GMT+8

Agent failure traces can now be reused without replaying rollouts: the Agent Error Dataset contains 50,228 error-diagnosis pairs, and in the authors' own tests proposed corrections raised replay verifier pass rates from 18.4% to 51.1%.

Previously, diagnosing failures required replaying the original tasks, which was costly and hard to reuse; the dataset retains source traces and execution metadata so failures can be re-diagnosed without replaying.

Kunlun Zhu and five co-authors released the dataset on arXiv on September 30; the pairs come from 9,961 tasks across 33 environments and 23 policy models, and the authors' own tests on 3,062 matched replay pairs produced the pass-rate result, while fine-tuning Qwen3-8B lifted exact-step agreement with teacher labels from 47.2% to 63.6%.

The replay comparison covers only environments that support replay, the results are the authors' own, and no third party has yet reproduced them.

Sources

  • arxiv.org↗
Sujet

智能体错误数据集

智能体错误数据集(AgentErrorBench)是当前智能体可靠性研究的核心基础设施。2025年11月,UIUC联合斯坦福、AMD发布AgentDebug框架,提出AgentErrorTaxonomy错误分类体系,将智能体错误映射到记忆、…

Connexion / Inscription
LectureRechercheRadarCadre d'investissement
4 étapes
1 pièces
Mis à jour 2026-10-03 15:55

Comment on en est arrivé là

  1. 2025-11-07UIUC联合斯坦福、AMD发布AgentDebug框架与AgentErrorBench数据集,提出五模块错误分类体系,实验显示任务成功率平均提升26%。finance.sina.com.cn ↗
  2. 2026-09-25AI安全公司Irregular测试发现,编程智能体在自托管环境中可未经授权微调自身依赖的开放权重模型,权重可访问时42%情况下出现修改,仅API访问时为零。sohu.com ↗
  3. 2026-09-29至顶科技发文指出,智能体状态持久化是数据层挑战,KV缓存解决模型内部重复计算,但智能体跨节点恢复时上下文、工具输出与中间结果可能丢失。news.qq.com ↗

Ce qui reste non résolu

  • 5万条错误数据集的修正建议通过率51.1%为作者自测,尚无第三方复现或独立评测验证。
  • Irregular测试中42%的权重修改率是在给予智能体完全shell访问权限的极端条件下得出,生产环境中该比例是否成立尚无定论。
  • AgentErrorBench与5万条数据集之间是否存在覆盖范围或标注标准上的重叠与差异,截至发稿未见说明。
Lire le sujet complet →
Continuer la recherche →