독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스주제 · CODA技能漂移协议중대1 분 소요DisconfirmAI 편집부

Preprint: Self-Improving Agents Lose Half Their Gains When Skill Banks Drift Post-Deployment

New research shows that evaluating AI agents only on their final training skill bank hides severe fragility; when skills change after deployment, performance drops significantly.

2026-10-16 12:00 GMT+8

Self-improving AI agents often learn by co-evolving a policy model with a library of skills. Current standard evaluations test the agent against the exact skill bank it was trained on, assuming this represents its capability. However, in real-world deployment, skills are frequently added, removed, or rewritten without retraining the underlying policy model.

A new preprint introduces the CO-evolution Drift Audit (CODA), an evaluation protocol that simulates this post-deployment skill drift. The researchers tested four different co-evolution methods across five benchmarks. They found that under worst-case drift conditions, success rates fell 5 to 13 percentage points below the peak scores measured on the original delivered bank.

This drop erased between 52% and 93% of the performance gains that the co-evolved skill bank provided over a baseline with no skills at all. The study argues that the shared acceptance rule in these methods ties competence too tightly to the specific training bank, making agents fragile once that bank changes. CODA proposes new metrics for consistency under edits and resistance to corrupted skills, shifting the goal from peak performance to robust harnessing of evolving skills.

출처

  • arxiv.org↗
주제
로그인 / 회원가입
독서리서치레이더투자 프레임워크

CODA技能漂移协议

阿里巴巴通义实验室等机构提出“协同进化陷阱”概念,指出当前自进化智能体在部署后若技能库发生漂移而模型未重训,性能将显著退化。为此,团队推出CODA(CO-evolution Drift Audit)评估协议,量化这一盲点。

2 단계0 건업데이트됨 2026-10-17 15:43

현재까지의 과정

  1. 2026-10-14arXiv预印本《Evolving Together, Failing Apart》提交,正式提出CODA协议以评估自进化智能体的技能漂移脆弱性。arxiv.org ↗
  2. 2026-10-17新闻摘要发布,强调仅按最终技能库评估会掩盖智能体在部署后因技能变化导致的性能大幅退化。arxiv.org ↗

아직 해결되지 않은 문제

  • CODA提出的‘一致性率’和‘抗腐蚀率’是否会成为下一代智能体评估的标准指标?
  • 是否存在不依赖重训即可自动适应技能漂移的新架构方案?
  • 算子归纳(Operator Induction)等严格验证方法能否完全替代或补充协同进化范式?
전체 주제 읽기 →
리서치 계속하기 →