MemTrace Preprint: State-Consistent Memory Boosts DeepSWE Pass@1 by 21.2 Points for Coding Agents
New memory system validates historical evidence against current repo state, significantly outperforming baselines in long-horizon coding tasks.
ImportanceMaterialEvidenceE2 unreplicatedWrite-upQuick
The MemTrace system improves the pass@1 metric on the DeepSWE benchmark by 21.2 percentage points.
Traditional coding agents often fail to reconstruct a consistent task state during long-horizon, multi-file tasks because context refreshes or repository changes invalidate earlier execution evidence.
MemTrace stores execution history as immutable 'Memory Traces' anchored to key information and checks the validity of historical evidence against the current repository state before restoring it. Under the Codex CLI framework, compared to all fully evaluated baselines, it improved the SWE-EVO Resolved Rate by 4.4 points and the SWE-Milestone Score by 17.8 points.
Data comes from author-run tests in a preprint (v2 updated Oct 8); independent reproduction has not yet been reported.