読書リサーチレーダー投資フレームワーク
ログイン / 新規登録
← 読解に戻る
ニュース重大2 分で読めるDisconfirmAI 編集部

SenseTime AI Infrastructure CTO Xuan Shanming: Agentic Inference Competition Hinges on Architectural Efficiency, Not Chip Count

Xuan Shanming, Vice President and CTO of SenseTime's Large Device AI infrastructure platform, presented the architectural approach for Agentic-era inference at the CSDI Summit in Shenzhen, China: layering inference by hardware requirements — compute-intensive Prefill on domestic Chinese chips, bandwidth-intensive Decode on high-end GPUs (PD separation); further exploring cross-datacenter KV Cache transfer for cross-domain PD separation, and splitting Attention and FFN within Decode onto different hardware (AF separation). Xuan claimed 5x throughput improvement under specific test conditions and projected 150%-200% single-card throughput gains from AF separation, but these figures are vendor self-reported with undisclosed test configurations.

2026-10-19 13:22 GMT+8

Agentic applications have pushed token consumption to thousand-fold levels while high-end chip supply cannot keep pace. Xuan Shanming, Vice President and CTO of SenseTime's Large Device platform, outlined the response at the October 16 CSDI Summit in Shenzhen, China: rather than simply stacking more compute, the approach decomposes the inference process by hardware requirements, letting different chips handle what they do best.

Large model inference has two phases: Prefill processes input tokens and is compute-intensive; Decode generates output tokens step by step, requiring continuous reads from context cache with high memory bandwidth demands. PD separation decouples these two steps — using compatible domestic Chinese compute for Prefill, high-bandwidth high-end GPUs for Decode, connected via KV Cache transfer. This isn't simply running different chips together; it's reorganizing compute resources around their characteristics. In some heterogeneous configurations, 30 to 50 mid-to-low-tier Prefill nodes may be needed to match one Decode node's processing capacity. Once a Decode node fails, large amounts of Prefill resources sit idle, necessitating dynamic failover and role-switching mechanisms.

Building on this foundation, Xuan described two extension directions. Cross-domain PD separation aims to break single-datacenter boundaries: local mid-to-low-tier compute handles Prefill, transferring KV Cache to high-end clusters in other regions for Decode. This becomes feasible as model architecture evolves — KV Cache size for 1M-context models has dropped from nearly 100GB to about 1GB in some cases, dramatically reducing cross-domain transfer pressure. AF separation further decomposes within the Decode phase: Attention handles context and consumes bandwidth, FFN performs matrix multiplications and consumes compute, splitting them onto different hardware for independent scaling.

ログイン / 新規登録
読書リサーチレーダー投資フレームワーク

Xuan stated that under specific test configurations, heterogeneous inference and resource coordination can increase overall token throughput to 5x the baseline; AF separation is projected to yield 150%-200% single-card throughput gains and 30%-40% operating cost reductions. These figures are SenseTime self-reported, with test configurations not publicly disclosed and no independent replication. Cross-domain PD separation and AF separation remain in exploratory stages. SenseTime's Large Device platform currently serves 6 trillion tokens daily, projected to reach 10 trillion daily by Q4 2026.

ソース

  • qbitai.com↗
リサーチを続ける →