CROP filters distillation tokens by counterfactual task relevance, gaining 1.92 and 2.96 points
CROP uses rewrite-calibrated counterfactual sensitivity to estimate token task relevance; the authors self-report gains of 1.92 and 2.96 points over the strongest non-CROP selector across two teacher-student distillation settings.
ImportanceLocalEvidenceE2 unreplicated
The selective online distillation method CROP concentrates supervision on response tokens relevant to the current input's semantic content: it uses rewrite-calibrated counterfactual sensitivity to estimate each token's task relevance, and the authors self-report aggregate performance gains of 1.92 and 2.96 points over the strongest non-CROP selector across two teacher-student distillation settings.
Previously, distillation supervision covered tokens indiscriminately, without selecting by task relevance; matched controls show CROP's selected points beat random and lowest-relevance selection.
The measurement is self-reported by the authors (first party), on two teacher-student distillation settings against the strongest non-CROP selector baseline, with gains of 1.92 and 2.96 points.
Results are author self-reported with no third-party replication yet; the paper was first submitted August 13 and updated to v3 on September 16 (arXiv:2608.13387).
Source: arXiv:2608.13387 abstract page ↗