Self-evolving agents' skill pools pollute past a critical size; VaG gating claims 72% pass@1 on Terminal-Bench 2
Self-reported by Linfang Shang and six co-authors: beyond a critical size new skills degrade performance, and their VaG gating lifts Terminal-Bench 2 to 72% pass@1.
ImportanceLocalEvidenceE2 unreplicated
When self-evolving agents distill skills from execution trajectories, new skills start to degrade performance once the skill pool passes a critical size — the core finding self-reported by Linfang Shang and six co-authors, whose Verifier-as-Gatekeeper (VaG) method claims a round-by-round rise to 72% pass@1 on Terminal-Bench 2.
Under unconditional skill accumulation, defective skills enter the decision context and become references for later distillation, forming a cross-round contamination chain; deleting the source skill afterwards recovers only a small fraction of the loss.
The authors therefore propose Verifier-as-Gatekeeper (VaG) gating, which filters skills one by one with three types of reviewers plus marginal-gain screening, and reports a skill pool about one-fifth the size of unconditional accumulation. These are first-party results.
Boundary: the results have not been reproduced by third parties; the preprint was submitted to arXiv on August 6 and updated as v2 on September 17 (2608.05810).