독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스간단한 요점1 분 소요DisconfirmAI 편집부

CROP filters distillation tokens by counterfactual task relevance, gaining 1.92 and 2.96 points

CROP uses rewrite-calibrated counterfactual sensitivity to estimate token task relevance; the authors self-report gains of 1.92 and 2.96 points over the strongest non-CROP selector across two teacher-student distillation settings.

2026-09-22 12:00 GMT+8

The selective online distillation method CROP concentrates supervision on response tokens relevant to the current input's semantic content: it uses rewrite-calibrated counterfactual sensitivity to estimate each token's task relevance, and the authors self-report aggregate performance gains of 1.92 and 2.96 points over the strongest non-CROP selector across two teacher-student distillation settings.

Previously, distillation supervision covered tokens indiscriminately, without selecting by task relevance; matched controls show CROP's selected points beat random and lowest-relevance selection.

The measurement is self-reported by the authors (first party), on two teacher-student distillation settings against the strongest non-CROP selector baseline, with gains of 1.92 and 2.96 points.

Results are author self-reported with no third-party replication yet; the paper was first submitted August 13 and updated to v3 on September 16 (arXiv:2608.13387).

Source: arXiv:2608.13387 abstract page ↗

출처

  • arxiv.org↗
리서치 계속하기 →
로그인 / 회원가입
독서리서치레이더투자 프레임워크