arXiv Preprint: New Frameworks Address 'Selection-to-Update' Gap in Self-Evolving LLMs, Boosting Math Scores by 5.3%
A new study identifies a flaw in self-evolving language model loops where seemingly learnable tasks don't yield useful updates. Proposed frameworks HeirCommit and EvoHeir reduce harmful inheritance and improve reasoning performance.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
Self-evolving language models must decide which updated model the next round inherits in every training cycle. Current systems like R-Zero and Agent0 typically make this decision based on whether a task looks learnable, inheriting all resulting updates. However, a new arXiv preprint titled "Selection Is Not Learning: Choosing Model Futures" identifies a critical "selection-to-update" gap in these loops: most updates on verified tasks fail to make correct answers more likely, yet existing loops inherit them all.
To address this, researchers propose two new frameworks. HeirCommit freezes the updated model and uses fresh evaluation questions to decide whether to inherit it or retain the current model. Analysis shows that at the same acceptance rate, HeirCommit cuts the harmful share of accepted updates by 68-83%. Another framework, EvoHeir, separates parameter inheritance from experience inheritance, using current and archived experience to repair failures and preserve successes before final confirmation.
Tests across four backbones show that EvoHeir achieves the best averages in mathematics and general reasoning, raising the math average by 5.3% relative to the strongest baseline, Agent0. The framework allows useful experience to persist even when the model that produced it is not inherited. Note that this is a preprint paper, and its results are based on author-run tests that have not yet been independently reproduced.