Latent Space AINews Analyzes DeepSeek's Open-Sourced Inference Engine DeepInfer: Details on 155T Daily Tokens and $38/GPU-Hour Revenue
This AINews episode deeply reports on Chinese AI firm DeepSeek's open-sourcing of its production-grade inference framework, DeepInfer. The program goes beyond technical architecture (such as disaggregated prefill/decode and SWA bounded replay) to analyze disclosed economic data: $38 revenue per GPU-hour at off-peak pricing, alongside community debates on derived margins and payback periods. It is a valuable resource for readers focused on AI infrastructure cost structures and commercial viability.
Latent Space's AINews column published an in-depth report on October 21, 2026, regarding DeepSeek's open-sourcing of its internal inference framework, DeepInfer. This episode is not merely a news recap but a dual-dimensional dissection of this "Chinese open-source bomb" event from both technical and economic perspectives. DeepSeek claims the framework powers its V4.1 Flash model's production service, achieving a single-cluster throughput of 155 trillion tokens per day and generating $38 in revenue per GPU-hour under off-peak pricing strategies.
The program extensively cites DeepSeek's engineering reports and community analyses, noting that these figures rely on highly optimized caching mechanisms—a 96% input cache hit rate implies most tokens do not require recomputation. However, the report honestly highlights limitations: the so-called "fully open source" status excludes critical KV-cache management and load-balancing code, as these components remain tightly coupled to DeepSeek's internal infrastructure. Furthermore, vLLM team's subsequent comparative data showed decode throughput within 6% of DeepInfer under specific configurations, somewhat challenging the notion that DeepSeek's efficiency is inaccessible to existing open-source engines.
For readers involved in AI infrastructure investment or engineering implementation, the core value of this episode lies in demonstrating how to reverse-engineer business model feasibility from public technical disclosures. The program aggregates estimation processes by various analysts (such as zephyr_z9, Teortaxes) regarding profit margins, emphasizing the impact of hardware costs, cache storage overheads, and unaccounted operational expenses on final profitability. Although no audited financial results are provided, this economic deduction based on first-hand technical data offers a rare perspective on understanding the true cost structure of large model services.