AI Inference Infrastructure Company Rebellions and Hanyang University Find FP4 Reasoning Failure Originates from Low-Entropy Position Sampling Errors; ReQAT Enables 4-bit Full Quantization to Match BF16 Precision
Research published by Rebellions and Hanyang University at ICML 2026 reveals that the mechanism by which FP4 quantization degrades reasoning models is not, as intuition suggests, perturbation at high-entropy branching points, but rather the cumulative effect of probability mass shifts in the model's most confident low-entropy positions under temperature sampling. The resulting ReQAT framework (Trace-Aligned QAT, Selective Entropy Minimization, Quantization-Friendly Initialization) enables R1-Qwen-14B to achieve 65.94% AIME accuracy under NVFP4 W4A4KV4, comparable to the 65.46% of BF16 full fine-tuning, while delivering a 3.05x throughput increase at 8K output lengths.
Hardware support for 4-bit floating-point formats has outpaced model tolerance. FP4 compute cores are now shipping, and microscaled formats like MXFP4 and NVFP4 make 4-bit floating-point activations feasible, yet it is precisely in reasoning models—those that generate ultra-long chains of thought to solve math problems—that precision collapses first. In an oral paper presented at ICML 2026, Rebellions and Hanyang University propose a counterintuitive explanation: FP4 damage to reasoning occurs primarily at the model's most confident positions, rather than at its most hesitant branching points.
The research team initially hypothesized that insufficient fine-tuning budget was the bottleneck. On R1-Llama-8B, they increased the number of fine-tuning epochs from one to five, yet the post-PTQ accuracy drop remained stuck between 11 and 15 points, indicating the problem lay not in training volume. They then turned to analyzing where quantization errors appeared within reasoning traces, tracking the entropy of the next-token distribution at each decoding step. Grouping over 1.5 million generated tokens by entropy revealed two populations: low-entropy tokens, comprising roughly 80% of the trace, consisting mainly of digits, variable names, and operators that the model predicts almost deterministically; and high-entropy tokens, such as discourse markers like 'Then,' 'Let,' and 'Hmm,' which appear at reasoning path branches.
Intuition suggests high-entropy positions are the sensitive ones—a small perturbation there could change the top-1 token. However, two controlled experiments yielded the opposite answer. The first experiment routed each prediction to either a BF16 or an FP4 model based on entropy: sending only low-entropy tokens to BF16 recovered most of the lost accuracy, while sending only high-entropy tokens provided little help. The second experiment injected Gaussian noise into logits as a proxy for quantization error: noise applied to the lowest-entropy 25% of positions caused a steep drop in AIME-2025 accuracy, whereas noise on the highest-entropy 25% had a much smaller effect. This pattern held across R1-Qwen-7B, R1-Qwen-14B, and the newer Qwen3-4B and Qwen3-8B.
The mechanism lies in the distribution tail. In low-entropy buckets, FP4 rarely changes the top-1 token, but the probability mass outside it grows significantly relative to BF16. Under greedy decoding this has little impact, but production services use temperature sampling, where every position carries a tiny probability of jumping to the wrong word. A reasoning trace spans thousands of positions; a single incorrect digit or operator typically invalidates the entire subsequent derivation. Perturbations at high-entropy positions, by contrast, mostly alter wording without changing the final answer. The team summarizes this finding: the dominant cause of FP4 reasoning failure is increased sampling error at the positions where the model is most confident, and standard QAT applies uniform loss across all tokens, failing to target this specific failure mode.
The ReQAT framework designs three components around this insight. Trace-Aligned QAT (TAQ) operates in two stages: first, fine-tune the model in BF16 on a reasoning dataset; second, enable quantization noise for QAT on a subset of the same traces used in Stage 1. Tracking token entropy during training revealed that standalone fine-tuning or standalone QAT primarily altered the high-entropy interval, whereas fine-tuning followed by QAT on the identical traces also changed the low-entropy interval—the model having already learned each trace in Stage 1 meant the Stage 2 gradients acted upon the low-entropy predictions made fragile by quantization. On R1-Qwen-14B under MXFP4 W4A16 with a 350M-token budget, two-stage training without trace alignment raised AIME accuracy from 62.29% to 62.60%, while trace alignment brought it to 67.29%.
Selective Entropy Minimization (SEM) adds an entropy minimization term to the SFT loss, applied only to the lowest-entropy 75% of tokens in each batch, with weight increasing as entropy decreases; the highest-entropy 25% are excluded to preserve the original distribution at branch points. TAQ restored most accuracy under W4A4, but accuracy dropped again when the KV cache was quantized to 4 bits, pointing to initialization issues in QAT. Q-FIT jointly calibrates two known transformations per layer—channel-pair scaling before RoPE and per-channel shifting after RoPE—to minimize the difference between BF16 and 4-bit KV attention outputs. Neither transformation alone yielded reliable gains in ablation studies, but joint calibration raised NVFP4 W4A4KV4 accuracy from 63.13% to 65.94%.
On R1-Qwen-14B, fine-tuned on the OpenThoughts3 math subset and evaluated on AIME-120 (AIME 2022–2025, averaged over 8 seeds), results under the same total budget were: BF16 baseline 56.83%, BF16 full fine-tuning 65.46%, direct PTQ under NVFP4 W4A4KV4 only 50.13%, standard QAT 58.86%, and ReQAT reaching 65.94%—slightly higher than BF16 full fine-tuning. Both BF16 fine-tuning and QAT saturated at approximately 280M tokens, whereas ReQAT continued to improve with budget, achieving the highest accuracy in all test settings, including comparisons against AWQ, QuaRot, FlatQuant, and QAD.
Matching BF16 accuracy means the throughput advantage of FP4 can be realized directly. Measuring end-to-end output-token throughput on a single NVIDIA B200 GPU with 1,000 requests of 512-token prompts, a batch limit of 256, and KV cache allocated to memory limits: at 2K output length, NVFP4 showed about 1.8x to 1.9x speedup over BF16, reflecting the raw compute advantage of 4-bit GEMM; at 8K output length, BF16 could no longer maintain the same batch size due to KV cache growth while NVFP4 could, yielding a speedup exceeding 3x. Even accounting for the 4% to 5% overhead of Q-FIT, ReQAT reached 3.05x. For long reasoning outputs, most of the FP4 gain comes from the larger batch sizes permitted by KV cache compression, making accuracy retention with a 4-bit KV cache a necessary condition for realizing FP4 efficiency on inference workloads.
The core insight of this work can be understood independently of specific models and formats: where the training signal is directed matters more than how much is thrown at it—increasing token count or epochs within the budget did not close the FP4 gap, whereas targeting low-entropy positions did. Code and the training recipe are publicly available at github.com/aiha-lab/ReQAT.