AWS: Sequence-Level Knowledge Distillation Compresses Shopping Agents to 4B Parameters, Preserving Reasoning Chains Is the Key to Success
An Amazon Web Services technical blog demonstrates how sequence-level knowledge distillation (SeqKD) transfers the behavior of frontier-model-driven shopping agents into a Qwen3.5-4B small model. The core finding: preserving the teacher's reasoning process in the training objective contributes approximately 13 percentage points more to task success rate than preserving tool calls alone; whether distillation succeeds depends on whether the task falls within the student's capability range, rather than the parameter gap between student and teacher.
The cost structure of agent applications differs from single-turn question answering. A shopping assistant completing one recommendation may undergo multiple steps—searching, reading results, narrowing scope, verifying constraints, and submitting an answer—in a multi-step loop, with each step constituting an independent billed call to a frontier model. At scale, this per-token billing loop makes deployment unviable. The solution offered by the AWS blog is sequence-level knowledge distillation (SeqKD): have the frontier teacher model generate complete multi-step trajectories, use a benchmark grader to filter for trajectories that actually solve the task, and then perform supervised fine-tuning on the small model using these trajectories. The teacher only needs to output text, requiring no weights, logits, or reinforcement learning infrastructure, so closed-source API models can also serve as teachers.
The most practically valuable finding in this blog is an ablation experiment: given the same 426 training problems and the same filtering pipeline, the only difference was whether the teacher's reasoning text was retained in the training objective. Removing reasoning and keeping only tool calls dropped the average absolute success rate from 0.541 to 0.415, a difference of about 13 percentage points. The authors explain that reasoning is not decorative narration but a causal chain connecting observations to next actions. Agents need to decide what to search for, how to filter, and when to terminate at each step, decisions driven by reasoning. If the student learns only the sequence of tool calls without the underlying reasoning, it achieves only partial transfer—preserving reasoning is the key to bringing the student close to the teacher's level.