LectureRechercheRadarCadre d'investissement
Connexion / Inscription
← Retour à la lecture
ActualitésSujet · 月之暗面Kimi模型系列Matériel1 min de lectureRédaction DisconfirmAI

vLLM Delivers Day-0 Support for Kimi K3.1 Pro: Sparse Attention and Packed KV Cache Boost Long-Context Inference Efficiency

vLLM provides day-0 support for Moonshot AI's Kimi K3.1 Pro, optimizing long-context inference on NVIDIA, AMD, and other hardware through sparse attention and a packed KV cache.

2026-11-04

Open-source inference engine vLLM announced on November 4, 2026, day-0 support for Moonshot AI's Kimi K3.1 Pro model. The model features 2.8 trillion total parameters (104 billion active) and a 1 million token context window, with its core architectural change being the introduction of Sparse Attention (DSA) across all 24 full-attention layers.

Key technical implementations by vLLM include: First, a packed KV cache mechanism that unifies MLA latents, indexer keys, and KDA states into contiguous blocks, supporting FP8 or NVFP4 quantization formats to significantly reduce memory footprint and improve serving density. Second, token-sharded prefill, which achieves up to 1.69x faster Time To First Token (TTFT) compared to head-sharded methods at 340K token lengths. Third, Decode Context Parallelism (DCP) combined with a sparse indexer, speeding up the attention module by 1.3–1.5x and achieving up to 375.9 tokens/s per user throughput on GB300 GPUs.

Additionally, vLLM introduced zero-mismatch Reinforcement Learning (RL) for the first time, ensuring bit-identical log-probs between trainer and rollout engines, maintaining KL divergence at zero during PPO training and enhancing RL stability. These optimizations have been validated on NVIDIA GB300/Rubin, AMD MI350 series, Google TPU, and Alibaba T-Head PPU.

Sources

Connexion / Inscription
LectureRechercheRadarCadre d'investissement
vllm.ai
↗
Sujet

月之暗面Kimi模型系列

月之暗面(Moonshot AI,中国)于2026年10月发布Kimi K3.1 Pro模型,并于11月5日在Hugging Face正式开源其2.8万亿参数权重。该模型采用MoE架构,支持百万上下文,在编码基准Terminal-Bench…

4 étapes4 piècesMis à jour 2026-11-07 04:12

Comment on en est arrivé là

  1. 2026-10-23月之暗面在API平台发布Kimi K3.1 Pro及Highspeed版本,承诺11月4日前发布完整模型权重。Simon Willison报道其定价与GPT-6.1 Sol持平,低于Claude Opus 5.5。simonwillison.net ↗
  2. 2026-10-24官方博客公布详细基准数据:Terminal-Bench 4.0得分65.7%(略低于Claude Opus 5.5的66.4%),新增3D资产生成能力,BenchCAD得分92.7%。kimi.ai ↗
  3. 2026-11-04vLLM团队宣布为Kimi K3.1 Pro提供首日支持,通过稀疏注意力(DSA)和打包KV缓存优化,在NVIDIA、AMD等硬件上实现高效长上下文推理。vllm.ai ↗

Ce qui reste non résolu

  • 自定义许可证中关于‘相关公司’营收合并计算的具体界定标准是否清晰?
  • 稀疏注意力机制在实际生产环境中对非vLLM框架的兼容性表现如何?
  • 开源权重后,社区微调版本是否会突破原模型在特定领域(如网络安全)的能力限制?

Également dans ce fil

  • Moonshot AI Releases Open Weights for Kimi K3.1 Pro with $20M Commercial Threshold2026-11-05
  • Moonshot AI Releases Kimi K3.1 Pro: Coding Benchmark Nears Claude Opus 5.5, Adds 3D Asset Generation2026-10-24
Lire le sujet complet →
Continuer la recherche →