NVIDIA: When to Use Switches Instead of Ring Topologies for Local AI Clusters
NVIDIA published benchmark data for an eight-node DGX Spark cluster and explained the performance boundary between switch and ring topologies: all-to-all switching reduces communication rounds and latency for small messages and few nodes; as message size or node count increases, bandwidth overhead rises, making ring forwarding more efficient. This assessment applies to local inference clusters based on RoCE interconnects.
At the GTC Berlin conference in October 2026, NVIDIA announced official support for eight-node DGX Spark clusters, scaling unified memory from 128 GB per node to 1 TB across eight nodes, enabling local desktop execution of frontier-class models such as GLM 5.3 (753B). However, more noteworthy than memory capacity is the topology selection mechanism presented in this article—it answers a practical engineering question: when building a local multi-node inference cluster, when is a switch necessary, and when is a direct ring connection sufficient?
NVIDIA tested a collective communication backend called RoCEnante on the eight-node cluster. Its principle is all-to-all exchange: each node sends its input directly to all other nodes, then performs reduction locally, with the entire process coordinated by a single GPU kernel. Compared to ring all-reduce, which requires data to hop between adjacent nodes over multiple rounds, all-to-all compresses communication into a single step, thereby reducing latency in tensor-parallel decoding scenarios characterized by small messages, low concurrency, and a modest number of nodes.
The tradeoff is greater data movement. Each node must send its complete input to every peer and perform reduction on all inputs locally. As message size or node count grows, this extra traffic becomes expensive, and the ring's advantage in bandwidth efficiency re-emerges. NVIDIA's test data confirms this boundary: using RoCEnante on a switch topology, eight-node decode throughput reached 52.58 TPS (a 4.43x speedup relative to the single-node baseline), whereas without RoCEnante on the same topology, the speedup was only 3.05x. At four nodes with NVFP4 precision, RoCEnante delivered approximately a 21% improvement in decode throughput; at eight nodes with non-quantized models, this improvement dropped to about 18%, consistent with the aforementioned latency-versus-bandwidth tradeoff.