Dynamic power allocation recovers 63% of throttled performance gap

Under a 30% total power cut, allocating power dynamically along each task's power-performance curve recovered an average of about 1,500 tokens/s/task, or 63% of the gap between even allocation and the theoretical optimum.

Previous throttled scheduling mostly used even allocation, leaving this gap because it did not exploit differences in tasks' power-performance curves.

The result comes from 131 H200 training runs, 24 validation runs and 34 matched H100 tasks, self-reported by the preprint's authors.

Experiments used at most 32 GPUs, concentrated on H100/H200; the preprint was submitted on September 10, has not been peer-reviewed, and has no third-party replication; they do not extrapolate to 10,000-GPU clusters, inference workloads or grid benefits.