vLLM Announces Day-0 Support for Thinking Machines' Inkling 2 Small Multimodal MoE Model, Achieving 630 Tokens/s Per User on GB200
vLLM provides Day-0 support for Thinking Machines' new Inkling 2 Small multimodal MoE model, optimizing kernels and using DSpark speculative decoding to achieve up to 630 tokens/s per user on GB200 GPUs.
Open-source inference engine vLLM announced Day-0 support for Thinking Machines' newly released Inkling 2 Small model on November 12, 2026. Inkling 2 Small is a multimodal (text and vision) Mixture-of-Experts (MoE) model with 594 billion total parameters and 17 billion active parameters, supporting a context window of up to 1 million tokens. The model also comes with a native DSpark speculator.
The vLLM team performed deep optimizations targeting NVIDIA Blackwell architecture GPUs (such as the GB200). Through techniques including kernel fusion, programmatic dependent launch (PDL), highly tuned Gluon attention kernels, and sequence parallelism, vLLM achieved significant performance improvements. According to official tests, single-user decoding speed reaches approximately 630 tokens/s at TP4 with an acceptance length of 3.5. Additionally, sequence parallelism optimization reduces the 8K-token prefill step by 22% at TP4 and 26% at TP8, freeing 1.3–1.9 GiB per GPU for KV cache.
Inkling 2 Small adopts new architectural features, including block attention residuals, gated attention, and mixed-precision MoE (NVFP4 for the first 40 layers, MXFP8 for the last 8). vLLM maintains the reference implementation's numerical precision in its integration and verified that accuracy remains consistent with non-speculative runs on benchmarks like NIAH, MMMU-Pro, and HLE when DSpark is enabled. Currently, vLLM also supports LoRA fine-tuning and reinforcement learning training for this model.