ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
TopicDisconfirmAI EditorialUpdated 2026-11-06 06:13

FP4推理精度失效

5 steps1 pieces5 Sources

AI推理基础设施公司Rebellions与韩国汉阳大学在ICML 2026发表研究指出，大推理模型（LRMs）在FP4全量化（W4A4KV4）下精度崩塌的主因并非高熵分支点扰动，而是模型最自信的低熵位置（如数字、运算符）在温度采样下的概率质量偏移累积。为此提出的ReQAT框架通过轨迹对齐QAT、选择性熵最小化及量化友好初始化，使R1-Qwen-14B在NVFP4 W4A4KV4下达到65.94%的AIME精度，匹配BF16全量微调水平，并在8K输出长度下实现3.05倍吞吐量提升。【背景】此前NVIDIA B200等硬件已原生支持FP4，但软件栈中频繁出现的精度异常（如vLLM和SGLang中的Bug报告）证实了“硬件能力超前于模型容忍度”的行业痛点。

How it got here

  1. 2025-09-10

    SGLang社区报告在NVIDIA B200上使用Flashinfer后端运行Llama-3.1-405B-Instruct-FP4时出现严重精度问题（GSM8K准确率降至0.015），而Triton后端正常。

    github.com ↗
  2. 2026-01-06

    vLLM社区报告在B200上结合DeepEP低延迟模式与特定DP Chunk Size配置时，NVFP4 MoE模型出现数值精度下降，疑似与CUDA Graphs有关。

    github.com ↗
  3. 2026-06-14

    Rebellions与汉阳大学团队向arXiv提交论文《ReQAT: Achieving Full-Precision Reasoning Accuracy with 4-bit Floating-Point Quantization-Aware Training》，提出解决FP4推理精度失效的新框架。

    arxiv.org ↗
  4. 2026-07-08

    该研究在ICML 2026会议上作为Oral Presentation展示，正式确立低熵位置采样误差为FP4失效主因的理论解释。

    icml.cc ↗
  5. 2026-11-05

    Rebellions官方博客发布文章《Can We Quantize Everything to 4 Bits?》，详细解读ReQAT机制及其对推理基础设施的意义。

    rebellions.ai ↗

What is still unsettled

  • ReQAT框架在其他非Qwen系列或更大参数规模（如70B+）的推理模型上的泛化能力如何？
Sign in / Sign up
ReadingResearchRadarInvestment framework
  • 除了低熵位置，激活分布中的极端离群值（Outliers）在FP4 KV Cache量化中是否仍构成不可忽略的次要误差源？
  • 随着NVIDIA Blackwell架构普及，主流推理引擎（vLLM/SGLang）是否会集成类似ReQAT的训练感知策略，还是仅依赖更激进的PTQ算法？

    • Sources

    • rebellions.aiRebellions Blog↗
    • icml.ccICML 2026 Poster↗
    • arxiv.orgarXiv Paper↗
    • github.comSGLang Issue #10284↗
    • github.comvLLM Issue #31840↗

    Coverage in this topic

    • AI Inference Infrastructure Company Rebellions and Hanyang University Find FP4 Reasoning Failure Originates from Low-Entropy Position Sampling Errors; ReQAT Enables 4-bit Full Quantization to Match BF16 Precision2026-11-06News

      Research published by Rebellions and Hanyang University at ICML 2026 reveals that the mechanism by which FP4 quantization degrades reasoning models is not, as intuition suggests, perturbation at high-entropy branching points, but rather the cumulative effect of probability mass shifts in the model's most confident low-entropy positions under temperature sampling. The resulting ReQAT framework (Trace-Aligned QAT, Selective Entropy Minimization, Quantization-Friendly Initialization) enables R1-Qwen-14B to achieve 65.94% AIME accuracy under NVFP4 W4A4KV4, comparable to the 65.46% of BF16 full fine-tuning, while delivering a 3.05x throughput increase at 8K output lengths.