ACT encoder ablation's 35%→2% drop fails to reproduce
Bo Kang re-ran ACT's CVAE encoder ablation on the original code and did not reproduce the reported drop in average success rate from 35% to 2%.
ImportanceLocalEvidenceE2 unreplicated
A re-run of the CVAE encoder ablation for the robot imitation learning method ACT did not reproduce the reported 35%→2% success-rate drop: as relayed by Bo Kang, the original paper claimed that removing the encoder dropped average success rate on two simulation tasks from 35% to 2%, but re-running on the original code did not reproduce the drop.
Previously readers had only the original paper's ablation conclusion, with no way to know that training duration and checkpoint selection alone can flip which policy wins, for unknown reasons.
The re-run was performed by Bo Kang on the original code (arXiv preprint, submitted September 15, revised September 16, id 2609.16745); the author also reports that the latent variable is zeroed at inference, that skipping the encoder improves training throughput, and has released code and evaluation tools.
The result is not peer-reviewed and has not been reproduced by others; it covers two simulation tasks only, not real-robot experiments.
arXiv abstract page (primary source) ↗、HTML full text v2 (primary source) ↗