vLLM Delivers Day-0 Support for Kimi K3.1 Pro: Sparse Attention and Packed KV Cache Boost Long-Context Inference Efficiency
vLLM provides day-0 support for Moonshot AI's Kimi K3.1 Pro, optimizing long-context inference on NVIDIA, AMD, and other hardware through sparse attention and a packed KV cache.
Open-source inference engine vLLM announced on November 4, 2026, day-0 support for Moonshot AI's Kimi K3.1 Pro model. The model features 2.8 trillion total parameters (104 billion active) and a 1 million token context window, with its core architectural change being the introduction of Sparse Attention (DSA) across all 24 full-attention layers.
Key technical implementations by vLLM include: First, a packed KV cache mechanism that unifies MLA latents, indexer keys, and KDA states into contiguous blocks, supporting FP8 or NVFP4 quantization formats to significantly reduce memory footprint and improve serving density. Second, token-sharded prefill, which achieves up to 1.69x faster Time To First Token (TTFT) compared to head-sharded methods at 340K token lengths. Third, Decode Context Parallelism (DCP) combined with a sparse indexer, speeding up the attention module by 1.3–1.5x and achieving up to 375.9 tokens/s per user throughput on GB300 GPUs.
Additionally, vLLM introduced zero-mismatch Reinforcement Learning (RL) for the first time, ensuring bit-identical log-probs between trainer and rollout engines, maintaining KL divergence at zero during PPO training and enhancing RL stability. These optimizations have been validated on NVIDIA GB300/Rubin, AMD MI350 series, Google TPU, and Alibaba T-Head PPU.