NVIDIA Says TF32x3 Accelerates Machine Learning Interatomic Potentials While Preserving Energy Conservation, Single-Pass TF32 Fails to Converge
Machine learning interatomic potentials are sensitive to numerical errors in forces, leading most to stick with FP32 inference. NVIDIA reported in a technical blog that BF16x9 and TF32x3—two Tensor Core methods for simulating FP32—can accelerate PET molecular dynamics steps by 1.1–1.7x and approximately 1.5x, respectively, without retraining the model. TF32x3 maintains physical behavior consistent with FP32 in energy conservation and structural relaxation convergence tests, whereas the faster single-pass TF32 fails to meet force convergence standards.
Machine learning interatomic potentials (MLIPs) use neural networks to predict atomic energies and forces at a fraction of the cost of quantum mechanical methods like density functional theory, but their precision requirements remain high. Forces are derivatives of energy with respect to atomic positions; in molecular dynamics simulations, forces at every step determine updates to atomic velocities and positions, meaning numerical errors accumulate along the trajectory. Therefore, even though GPUs offer much higher throughput for low-precision matrix multiplication than for FP32, many MLIPs continue to rely on FP32 inference.
In a technical blog post, NVIDIA evaluated two Tensor Core FP32 simulation methods. On Blackwell GPUs, BF16x9 decomposes each FP32 operand into three scaled BF16 components, computes nine pairwise products, and accumulates them in FP32; it can be enabled via the environment variable CUBLAS_EMULATE_SINGLE_PRECISION=1 without modifying model code. On Ampere GPUs, TF32x3 splits each FP32 operand into a leading TF32 component and a residual TF32 component, computing three cross-products (leading × leading, leading × residual, residual × leading) while omitting the smallest residual × residual term. Implemented via CUTLASS's OpMultiplyAddFastF32 operator, inputs and outputs remain in FP32, making the method transparent to the rest of the model.