독서리서치레이더투자 프레임워크
로그인 / 회원가입
← 읽기로 돌아가기
뉴스중대1 분 소요DisconfirmAI 편집부

vLLM Announces Day-0 Support for Thinking Machines' Inkling 2 Small Multimodal MoE Model, Achieving 630 Tokens/s Per User on GB200

vLLM provides Day-0 support for Thinking Machines' new Inkling 2 Small multimodal MoE model, optimizing kernels and using DSpark speculative decoding to achieve up to 630 tokens/s per user on GB200 GPUs.

2026-11-12

Open-source inference engine vLLM announced Day-0 support for Thinking Machines' newly released Inkling 2 Small model on November 12, 2026. Inkling 2 Small is a multimodal (text and vision) Mixture-of-Experts (MoE) model with 594 billion total parameters and 17 billion active parameters, supporting a context window of up to 1 million tokens. The model also comes with a native DSpark speculator.

The vLLM team performed deep optimizations targeting NVIDIA Blackwell architecture GPUs (such as the GB200). Through techniques including kernel fusion, programmatic dependent launch (PDL), highly tuned Gluon attention kernels, and sequence parallelism, vLLM achieved significant performance improvements. According to official tests, single-user decoding speed reaches approximately 630 tokens/s at TP4 with an acceptance length of 3.5. Additionally, sequence parallelism optimization reduces the 8K-token prefill step by 22% at TP4 and 26% at TP8, freeing 1.3–1.9 GiB per GPU for KV cache.

Inkling 2 Small adopts new architectural features, including block attention residuals, gated attention, and mixed-precision MoE (NVFP4 for the first 40 layers, MXFP8 for the last 8). vLLM maintains the reference implementation's numerical precision in its integration and verified that accuracy remains consistent with non-speculative runs on benchmarks like NIAH, MMMU-Pro, and HLE when DSpark is enabled. Currently, vLLM also supports LoRA fine-tuning and reinforcement learning training for this model.

출처

로그인 / 회원가입
독서리서치레이더투자 프레임워크
vllm.ai
↗
리서치 계속하기 →