LesenForschungRadarAnlageframework
Anmelden / Registrieren
← Zurück zur Lektüre
NewsKurzfassung1 Min. LesezeitDisconfirmAI Redaktion

CROP filters distillation tokens by counterfactual task relevance, gaining 1.92 and 2.96 points

CROP uses rewrite-calibrated counterfactual sensitivity to estimate token task relevance; the authors self-report gains of 1.92 and 2.96 points over the strongest non-CROP selector across two teacher-student distillation settings.

2026-09-22 12:00 GMT+8

The selective online distillation method CROP concentrates supervision on response tokens relevant to the current input's semantic content: it uses rewrite-calibrated counterfactual sensitivity to estimate each token's task relevance, and the authors self-report aggregate performance gains of 1.92 and 2.96 points over the strongest non-CROP selector across two teacher-student distillation settings.

Previously, distillation supervision covered tokens indiscriminately, without selecting by task relevance; matched controls show CROP's selected points beat random and lowest-relevance selection.

The measurement is self-reported by the authors (first party), on two teacher-student distillation settings against the strongest non-CROP selector baseline, with gains of 1.92 and 2.96 points.

Results are author self-reported with no third-party replication yet; the paper was first submitted August 13 and updated to v3 on September 16 (arXiv:2608.13387).

Source: arXiv:2608.13387 abstract page ↗

Quellen

  • arxiv.org↗
Recherche fortsetzen →
Anmelden / Registrieren
LesenForschungRadarAnlageframework