AMD ROCm Team Proposes Configuration Pinning by Kernel Text as Key, Control Experiments Show Glue Acceleration Comes from Configuration Rather Than Kernel Body Replacement
The AMD ROCm team has moved the TorchInductor glue kernel configuration search out of the compilation process, using the SHA-256 fingerprint of the kernel text as a key to write configurations into the package. On two production models, pinned configurations achieved speedups of 1.388x and 2.846x relative to default startup; control experiments that placed the same configuration back into the compiler's kernel body reproduced the performance, indicating that the acceleration stems from the configuration rather than kernel body replacement.
When compiling inference graphs, TorchInductor generates a list of candidate configurations for each glue kernel (pointwise, reduction, persistent-reduction). After autotune timing, it writes the winner into a cache keyed by fusion number. This mechanism has two critical flaws: first, it is a closed choice, timing only the candidates listed by the compiler; if a faster tile is not on the list, it will never be selected. Second, the cache key is the fusion number rather than the kernel content; when an upstream template changes, the numbers rearrange, invalidating the cache. Additionally, another process on the same GPU can make the fastest configuration appear slower. In a ROCm blog post dated November 17, 2026, Yuchen Lin, Huan Li, Chunhung Wang, and Chao Chen proposed moving the search outside the compilation phase: offline scanning of candidate configurations, retaining only those that pass numerical checks and are confirmed to be faster via tracing. The final selection uses the SHA-256 fingerprint of the kernel text (with numeric suffixes removed and truncated to 20 hexadecimal characters) as the key, writing both the selected configuration and the kernel text into the AOTInductor package. At startup, there is no timing of candidates; the pre-verified configuration is loaded directly.
The core validation comes from a control experiment: placing the configuration pin file from the Tuned package back onto the Inductor compiler kernel body (without replacing any kernel source code). On GPU 6 running Model A, the glue time measured was 1567.2 us, essentially consistent with the Tuned result of 1598.7 us. On Model B, the compiler kernel body combined with the same configuration pin file yielded 391.2 us, compared to 350.88 us for Tuned. Latency figures also aligned closely. This demonstrates that the 1.388x and 2.846x speedups are almost entirely due to the configuration search; the AITER kernel body replacement (which involved only three SiLU kernels on Model A) is not a necessary condition.