AI Inference Infrastructure Company Rebellions and Hanyang University Find FP4 Reasoning Failure Originates from Low-Entropy Position Sampling Errors; ReQAT Enables 4-bit Full Quantization to Match BF16 Precision
Hardware support for 4-bit floating-point formats has outpaced model tolerance. FP4 compute cores are now shipping, and microscaled formats like MXFP4 and NVFP4 make 4-bit floating-point activations feasible, yet it is precisely in reasoning models—those that generate ultra-long chains of thought to solve math problems—that precision collapses first. In an oral paper presented at ICML 2026, Rebellions and Hanyang University propose a counterintuitive explanation: FP4 damage to reasoning occurs primarily at the model's most confident positions, rather than at its most hesitant branching points.
The research team initially hypothesized that insufficient fine-tuning budget was the bottleneck. On R1-Llama-8B, they increased the number of fine-tuning epochs from one to five, yet the post-PTQ accuracy drop remained stuck between 11 and 15 points, indicating the problem lay not in training volume. They then turned to analyzing where quantization errors appeared within reasoning traces, tracking the entropy of the next-token distribution at each decoding step. Grouping over 1.5 million generated tokens by entropy revealed two populations: low-entropy tokens, comprising roughly 80% of the trace, consisting mainly of digits, variable names, and operators that the model predicts almost deterministically; and high-entropy tokens, such as discourse markers like 'Then,' 'Let,' and 'Hmm,' which appear at reasoning path branches.
Intuition suggests high-entropy positions are the sensitive ones—a small perturbation there could change the top-1 token. However, two controlled experiments yielded the opposite answer. The first experiment routed each prediction to either a BF16 or an FP4 model based on entropy: sending only low-entropy tokens to BF16 recovered most of the lost accuracy, while sending only high-entropy tokens provided little help. The second experiment injected Gaussian noise into logits as a proxy for quantization error: noise applied to the lowest-entropy 25% of positions caused a steep drop in AIME-2025 accuracy, whereas noise on the highest-entropy 25% had a much smaller effect. This pattern held across R1-Qwen-7B, R1-Qwen-14B, and the newer Qwen3-4B and Qwen3-8B.
The mechanism lies in the distribution tail. In low-entropy buckets, FP4 rarely changes the top-1 token, but the probability mass outside it grows significantly relative to BF16. Under greedy decoding this has little impact, but production services use temperature sampling, where every position carries a tiny probability of jumping to the wrong word. A reasoning trace spans thousands of positions; a single incorrect digit or operator typically invalidates the entire subsequent derivation. Perturbations at high-entropy positions, by contrast, mostly alter wording without changing the final answer. The team summarizes this finding: the dominant cause of FP4 reasoning failure is increased sampling error at the positions where the model is most confident, and standard QAT applies uniform loss across all tokens, failing to target this specific failure mode.
The ReQAT framework designs three components around this insight. Trace-Aligned QAT (TAQ) operates in two stages: first, fine-tune the model in BF16 on a reasoning dataset; second, enable quantization noise for QAT on a subset of the same traces used in Stage 1. Tracking token entropy during training revealed that standalone fine-tuning or standalone QAT primarily altered the high-entropy interval, whereas fine-tuning followed by QAT on the identical traces also changed the low-entropy interval—the model having already learned each trace in Stage 1 meant the Stage 2 gradients acted upon the low-entropy predictions made fragile by quantization. On R1-Qwen-14B under MXFP4 W4A16 with a 350M-token budget, two-stage training without trace alignment raised AIME accuracy from 62.29% to 62.60%, while trace alignment brought it to 67.29%.
Sources:Rebellions