NVIDIA Technical Blog Details FLUX 3 Image Klein Blackwell Inference Optimization Methods, Single-GPU Speedup Up to 4.45x
The NVIDIA Technical Blog published a tutorial on optimizing inference for FLUX 3 Image Klein on Blackwell GPUs, employing four layers of optimization: NVFP4 quantization, fused GEMM kernels, quantized and sparse attention, step caching, and multi-GPU parallelism. These optimizations reduced text-to-image latency from 22.36 seconds to 6.46 seconds and image-to-image latency from 54.64 seconds to 12.28 seconds on a single B200 GPU, with quality loss quantified using LPIPS scores at each stage.
Generating high-resolution images with diffusion models requires dozens of sequential forward passes through a model with billions of parameters, where the computational cost of attention increases with resolution. The NVIDIA Technical Blog recently released a tutorial addressing inference latency for FLUX 3 Image Klein on Blackwell GPUs, proposing a layered optimization scheme. Each layer targets different computational bottlenecks in the pipeline; when stacked, these optimizations compress the text-to-image latency for the Base model at 2048×2048 resolution from 22.36 seconds to 6.46 seconds, and reduce image-to-image latency from 54.64 seconds to 12.28 seconds on a single B200 GPU.
The inference pipeline for FLUX 3 Image Klein consists of three components: Qwen3-VL-4B-Instruct handles text condition encoding, the Diffusion Transformer (DiT) executes denoising iterations, and an autoencoder decodes latent variables into pixel images. At 2048×2048 resolution, the denoising phase accounts for 96.8% of the total latency for the Base model, with attention computation occupying approximately 60% of the DiT GPU kernel time. Consequently, optimization focuses on two core bottlenecks: GEMM matrix multiplication throughput and attention computational costs.
The first layer combines NVFP4 quantization with PrimsTS fused GEMM kernels. NVIDIA states that NVFP4 delivers up to 1.62x faster text-to-image generation on a single B200 compared to BF16. The PrimsTS kernel fuses GEMM with SwiGLU, QKV projection, and RoPE rotary position encoding epilogues, reducing kernel launch overhead. The second layer addresses the attention bottleneck: regarding quantized attention, VC-Attention FP8 (with a repair budget ρ=0.005) provides a 1.18x speedup with an LPIPS score of 0.086; regarding sparse attention, Sol-Attention (τ=1.0, 12 initial dense steps) yields a 1.30x denoising speedup but raises the LPIPS to 0.151. The third layer, Cache-DiT, reuses intermediate Transformer outputs across adjacent denoising steps, nearly halving the Base latency (1.94x speedup) at a threshold of 0.24, with only an LPIPS increase of 0.033.