LesenForschungRadarAnlageframework
Anmelden / Registrieren
← Zurück zur Lektüre
NewsThema · DeepSeek推理框架Wesentlich2 Min. LesezeitDisconfirmAI Redaktion

DeepSeek Open-Sources DeepInfer, Its Production Inference Framework Supporting Nvidia Blackwell and Huawei Ascend

DeepSeek has open-sourced DeepInfer, its production-grade inference framework featuring speculative decoding and prefill optimizations, with support for Nvidia Blackwell and Huawei Ascend 950 chips, though some core scheduling code remains closed.

2026-10-21 07:11 GMT+8

Chinese AI startup DeepSeek on October 20, 2026, open-sourced DeepInfer, the inference framework powering its own production services. The framework aims to solve the challenge of serving large models to a massive user base at a reasonable cost. According to DeepSeek, one cluster currently serves 155 trillion tokens per day, generating approximately $38 in revenue per GPU per hour at off-peak pricing.

For now, DeepInfer supports only a single model, DeepSeek-V4.1-Flash, defaulting to the updated 1017 version released over the weekend. The framework is compatible with Nvidia Corp.'s SM100 Blackwell GPUs and Huawei Technologies Co. Ltd.'s Ascend 950 neural processing units, but does not support Hopper or other older Nvidia architectures. Two features ensuring identical outputs across runs work only on Nvidia hardware. All published performance benchmarks were run on Nvidia chips, and the figures are self-reported by DeepSeek.

Technically, DeepInfer employs a speculative decoding system called DSpark, where a draft model proposes five tokens at a time and a confidence head predicts which ones the main model will accept. This reduces the number of draft tokens checked per step by about 30%, yielding an estimated 4% to 7% throughput gain at production load. A second technique claims to nearly halve total prefill computation by running only the last 128 tokens of each prompt back through the model's decoder layers.

DeepSeek has not yet released its key-value cache management and load-balancing code, stating these are tightly coupled to internal infrastructure. Splitting prefill and decode across separate machines is also left to operators. DeepInfer is available on GitHub under the Apache 2.0 license, accompanied by a deepseek-recipe library that converts requests into token sequences using the same logic as DeepSeek's official API.

Anmelden / Registrieren
LesenForschungRadarAnlageframework

Quellen

  • siliconangle.com↗
Thema

DeepSeek推理框架

中国AI公司DeepSeek于2026年10月20日正式开源其内部生产级推理框架DeepInfer及配套组件deepseek-recipe,旨在降低大规模模型部署的算力与能源成本。该框架支持英伟达Blackwell(SM100)和华为昇腾9…

4 Schritte5 BeiträgeAktualisiert 2026-11-07 19:16

Wie es dazu kam

  1. 2026-10-20DeepSeek宣布开源内部推理框架DeepInfer及编解码组件deepseek-recipe,支持英伟达与华为平台。ithome.com ↗
  2. 2026-10-20SiliconANGLE报道DeepSeek开源DeepInfer,指出部分核心调度代码暂未公开,并提及美国国土安全部此前要求联邦机构移除DeepSeek产品。siliconangle.com ↗
  3. 2026-10-21Latent Space发布AINews深度解析,披露DeepSeek声称的单集群日吞吐155万亿Token及每GPU小时38美元营收数据,引发关于利润率与回本周期的争议。latent.space ↗

Was noch ungeklärt ist

  • DeepInfer未公开的KV缓存管理和负载均衡代码何时能完全解耦并开源?
  • vLLM等现有开源引擎能否通过配置优化达到DeepInfer宣称的效率水平?
  • DeepSeek披露的经济数据(如38美元/GPU小时营收)在不同工作负载和硬件成本下的实际利润率是多少?

Auch in diesem Thread

  • Latent Space AINews Analyzes DeepSeek's Open-Sourced Inference Engine DeepInfer: Details on 155T Daily Tokens and $38/GPU-Hour Revenue2026-10-23
  • DeepSeek releases open-source inference framework Deepinfer to cut self-hosting costs2026-10-22
Vollständiges Thema lesen →
Recherche fortsetzen →