ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
TopicDisconfirmAI EditorialUpdated 2026-11-20 09:44

AMD智能体推理调度

5 steps1 pieces5 Sources

AMD ATOM团队针对智能体编码工作负载(AgentX)的长历史复用与突发子代理特性,提出了三层服务优化策略。核心在于PrefillDelayer调度机制:在16并发场景下,该策略通过延迟预填充请求以合并短批次,使活跃会话的P90 token间延迟改善22%,但代价是新请求的中位首token时间(TTFT)恶化49%。这一权衡仅在特定并发区间启用,旨在平衡持续生成体验与新请求响应速度。此外,AMD结合Moonshot AI重建了基于Kimi K3/K2.6模型的端到端服务栈,利用统一内存带宽池(UMBP)实现多级KV缓存感知路由,进一步降低p99 TTFT并提升吞吐量。

How it got here

  1. 2026-05-14

    Red Hat与AMD宣布合作,推出支持AMD Instinct MI355X GPU的Red Hat AI 3.4,强调智能体AI需要新的基础设施栈,包括GPU用于复杂推理和CPU用于编排。

    redhat.com ↗
  2. 2026-07-21

    AMD发布技术文章,介绍与Moonshot AI合作重建的智能体AI服务栈,提出UMBP(统一内存与带宽池)概念,解决KV缓存跨层级调度问题。

    amd.com ↗
  3. 2026-09-03

    AMD发布PACE平台感知计算引擎,将其从纯推理引擎升级为智能体AI编排器,支持LangGraph作为原生编排层,实现确定性执行与基准测试。

    amd.com ↗
  4. 2026-10-09

    AMD展示TokenSpeed引擎在MI355X上运行Kimi K3的性能,指出在高并发下ATOM采用prefill coalescing策略会显著增加TTFT但改善TPOT。

    amd.com ↗
  5. 2026-11-18
Sign in / Sign up
ReadingResearchRadarInvestment framework

AMD ROCm博客详细解释ATOM团队的PrefillDelayer调度策略,量化其在16并发下对P90 interactivity(+22%)和中位TTFT(-49%)的具体影响。

rocm.blogs.amd.com ↗

What is still unsettled

  • PrefillDelayer策略在不同硬件配置或模型规模下的最优并发阈值是否一致?
  • UMBP多级缓存感知路由在实际生产环境中对总拥有成本(TCO)的具体影响数据尚未公开。
  • AMD PACE与ATOM/TokenSpeed等GPU加速引擎在混合部署时的协同调度效率如何?

Sources

  • rocm.blogs.amd.comROCm Blogs: Keep the Agent Warm↗
  • amd.comAMD Developer: Rebuilding Agentic AI for AMD GPU↗
  • amd.comAMD Developer: Kimi K3 on AMD Instinct GPUs with TokenSpeed↗
  • redhat.comRed Hat Blog: Agentic AI demands a new infrastructure stack↗
  • amd.comAMD Developer: Agentic Flow with AMD PACE↗

Coverage in this topic

  • AMD Demonstrates Trade-offs in Agent Inference Scheduling: Active Session Latency Drops 22%, New Request Wait Increases 49%2026-11-18News

    The ATOM team at Advanced Micro Devices (AMD) explains the three-layer service challenges of agent coding workloads, highlighting the scheduling strategy of PrefillDelayer: it improves P90 token inter-token latency by 22% under 16 concurrent sessions, at the cost of a 49% deterioration in median time-to-first-token for new requests, and explains why this strategy is only enabled within specific concurrency ranges.