LeituraPesquisaRadarFramework de investimento
Entrar / Cadastrar
← Voltar para a leitura
NotíciasTópico · AMD ROCmMaterial2 min de leituraEditorial DisconfirmAI

AMD ROCm Team Proposes Configuration Pinning by Kernel Text as Key, Control Experiments Show Glue Acceleration Comes from Configuration Rather Than Kernel Body Replacement

The AMD ROCm team has moved the TorchInductor glue kernel configuration search out of the compilation process, using the SHA-256 fingerprint of the kernel text as a key to write configurations into the package. On two production models, pinned configurations achieved speedups of 1.388x and 2.846x relative to default startup; control experiments that placed the same configuration back into the compiler's kernel body reproduced the performance, indicating that the acceleration stems from the configuration rather than kernel body replacement.

2026-11-17 08:00 GMT+8

When compiling inference graphs, TorchInductor generates a list of candidate configurations for each glue kernel (pointwise, reduction, persistent-reduction). After autotune timing, it writes the winner into a cache keyed by fusion number. This mechanism has two critical flaws: first, it is a closed choice, timing only the candidates listed by the compiler; if a faster tile is not on the list, it will never be selected. Second, the cache key is the fusion number rather than the kernel content; when an upstream template changes, the numbers rearrange, invalidating the cache. Additionally, another process on the same GPU can make the fastest configuration appear slower. In a ROCm blog post dated November 17, 2026, Yuchen Lin, Huan Li, Chunhung Wang, and Chao Chen proposed moving the search outside the compilation phase: offline scanning of candidate configurations, retaining only those that pass numerical checks and are confirmed to be faster via tracing. The final selection uses the SHA-256 fingerprint of the kernel text (with numeric suffixes removed and truncated to 20 hexadecimal characters) as the key, writing both the selected configuration and the kernel text into the AOTInductor package. At startup, there is no timing of candidates; the pre-verified configuration is loaded directly.

Entrar / Cadastrar
LeituraPesquisaRadarFramework de investimento

The core validation comes from a control experiment: placing the configuration pin file from the Tuned package back onto the Inductor compiler kernel body (without replacing any kernel source code). On GPU 6 running Model A, the glue time measured was 1567.2 us, essentially consistent with the Tuned result of 1598.7 us. On Model B, the compiler kernel body combined with the same configuration pin file yielded 391.2 us, compared to 350.88 us for Tuned. Latency figures also aligned closely. This demonstrates that the 1.388x and 2.846x speedups are almost entirely due to the configuration search; the AITER kernel body replacement (which involved only three SiLU kernels on Model A) is not a necessary condition.

Applicability boundaries include: fixed shapes, ensuring that one measured configuration corresponds to the production launch; this method is ineffective when the hot path remains an untuned GEMM or attention operation; pinning loses meaning when shapes change at every step; and it offers no help when the time bottleneck is launch overhead that the kernel body cannot fuse away. All data represents self-reported results from AMD in an environment featuring GFX942 (80 CUs), ROCm 7.2.2, and PyTorch 2.9.1. For Model A, the variance among three open-list compilation results reached 245 us, which exceeds the advantage Tuned holds over the fastest compilation; therefore, this advantage is not listed as a speedup ratio.

Fontes

  • rocm.blogs.amd.comROCm Blogs↗
Tópico

AMD ROCm

AMD ROCm 团队提出以内核文本 SHA-256 指纹为键的配置钉定法,将 TorchInductor 的 glue 内核配置搜索移出编译过程。在两个生产模型上,钉定配置相对默认启动获得 1.388x 和 2.846x 加速;控制实验表…

5 etapas1 peçasAtualizado 2026-11-20 11:37

Como chegamos aqui

  1. 2025-03-27GitHub PR #2008 提交,涉及 Triton/inductor 相关优化,包括更新 triton pin 至性能优化 commit 及添加 flex_attention 的实验性穷举调优模式。github.com ↗
  2. 2026-08-31GitHub PR #195658 开始推进 Composable Kernel (CK) pin 的更新,并生成 ck/config.h 以支持 CK wheel 构建。github.com ↗
  3. 2026-11-17AMD ROCm 官方博客发布文章《An AITER Pin Keyed by Kernel Text》,详细介绍以内核文本为键的配置钉定法及其在 Model A/B 上的性能测试结果。rocm.blogs.amd.com ↗

O que ainda está indefinido

  • 该配置钉定法在动态形状或频繁变更的模型架构中的维护成本如何?
  • AITER 库中其他未提及的内核类型是否也适用此指纹匹配机制?
  • 社区反馈中是否有针对 SHA-256 截断哈希碰撞风险的讨论?
Ler o tópico completo →
Continuar pesquisando →