vLLM Announces Day-0 Support for Recursive's Flint Model: ~500 Tokens/s Decoding on Single GB200 with 100K Context
vLLM achieves high-speed long-context inference for the 40B-parameter MoE model Flint on a single GB200 GPU through custom kernel optimizations.
Open-source inference engine vLLM announced on October 21, 2026, that it has achieved Day-0 support for Flint, a model released by Recursive. Flint is a Mixture-of-Experts (MoE) model with 40 billion total parameters but only about 2.5 billion active per token, featuring sliding-window attention in most layers and supporting a 262K-token context window.
According to technical details disclosed in the vLLM official blog, this support involves deep kernel optimizations targeting NVIDIA Blackwell architecture GPUs (such as the GB200). On a single GB200 GPU, decoding speed for a single request with 100K tokens of context reaches approximately 500 tokens/s. This performance is attributed to vLLM's replacement of multiple generic kernels in standard layers, including fused Q/K normalization, RoPE, and KV-cache write operations, as well as specialized optimizations for small-batch MoE routing and expert GEMV operations.
The Flint model ships with two checkpoint versions: NVFP4 (quantization-aware trained) and BF16. vLLM notes that its implemented NVFP4 path uses weight-only quantization (W4A16) and losslessly repacks GGUF files at load time, ensuring accuracy matches Recursive's reference implementation on benchmarks like GSM8K and GPQA-Diamond. Currently, these optimizations primarily target TP=1 (single-GPU tensor parallelism) configurations; other configurations fall back to vLLM's standard layers.