ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

CoreWeave's Min Jun: Agentic Traffic Breaks Network Assumptions Never Tested by Training

CoreWeave blogger Min Jun explains why the serial, multi-hop traffic pattern of agentic AI—where latency is the sum of every hop rather than the worst one—forces network design choices in topology, load balancing, and resilience strategies that differ fundamentally from those required for training.

2026-10-23 00:27 GMT+8

When training large models, networks face a rhythmic problem: thousands of GPUs simultaneously execute the same collective communication, over and over, for weeks. Bandwidth pressure is immense, but the pattern is predictable; engineers can design around this rhythm.

In a blog post published in August 2026, Min Jun of CoreWeave, an AI cloud infrastructure company, described another type of traffic: agents break tasks into steps, call retrievers, call tools, call another model, sometimes call another agent, and wait for each step to return before knowing what the next one is. This serial path is an agent chain, and its latency is the sum of every hop, not the worst single hop.

This distinction seems minor, yet it changes the logic of entire network design. When thousands of such chains run on the same hardware alongside training tasks and standard inference, traffic no longer resembles batch jobs; it becomes a ubiquitous, latency-sensitive, multi-hop conversation. Assumptions never tested by training begin to break.

The first consequence Min Jun explains is that oversubscription is not an option. When leaf switches have less uplink capacity to the spine than the nodes below them can generate, this gap remains invisible until two types of traffic compete for the same link. While training alone can tolerate some degree of oversubscription, a network mixing synchronized training bursts with thousands of small, sequential agent requests must simultaneously satisfy high aggregate bandwidth and low, predictable per-flow latency; oversubscribed links cannot achieve both. CoreWeave’s approach is to maintain a 1:1 non-blocking ratio at every layer.

Sign in / Sign up
ReadingResearchRadarInvestment framework

The second consequence involves how the network scales. Traditional two-tier leaf-spine topologies eventually exhaust switch radix limits; the solution is adding a third tier, but this increases hop count—which is precisely what agent chains cannot afford. CoreWeave adopts a multi-plane topology: distributing each GPU’s SuperNIC connections across two or more independent network planes, trading depth for width. Min Jun states this allows a flat, two-tier design to scale to 128,000 GPUs, 64 times the coverage of a single-plane network, without requiring redesign.

The third consequence is that load balancing must move from software to hardware. Traditionally handled by NCCL at the software layer to disperse collective communication traffic across available links, this approach carries two costs: competing with host CPUs for cycles, and only reacting to congestion after the fact rather than proactively routing around it as conditions change. CoreWeave uses NVIDIA ConnectX-9 SuperNICs to offload this task to hardware, allowing dedicated hardware paths to respond to real-time congestion telemetry at line rate. In its NVIDIA Vera Rubin NVL72 deployment, each GPU’s SuperNIC runs at up to 1,600 Gb/s; switches coordinate directly with SuperNICs, dispersing traffic across every available path, plane, and rail using real-time telemetry rather than static hashing.

The fourth consequence is that resilience cannot be centralized. A training job stopping for a few minutes is expensive, but it can recover from checkpoints. An agent stuck halfway through a ten-step chain, multiplied across thousands of concurrent chains, has no equivalent recovery point. Min Jun describes building resilience across five layers: the topology layer shifts traffic off degraded paths using multi-plane and multi-rail designs; the silicon layer uses BlueField-3 DPUs to place networking, isolation, and policy enforcement on dedicated hardware, freeing host CPU cycles; the software layer uses CoreWeave Kubernetes Service (CKS) to completely remove the hypervisor, ensuring virtualization faults do not become tenant issues; the operations layer uses Mission Control to automatically replace nodes once they cross health thresholds; and the geographic layer links multiple data centers into a single backbone, ensuring cross-site workloads do not depend on any single facility.

The fifth consequence is that observability must reach every hop. If the slowest hop is hidden, all previous design efforts are futile. CoreWeave uses NVIDIA network observability tools to trace flow-level performance from GPU to SuperNIC, transforming the vague symptom of “this agent chain feels slow” into specific links or queues that engineers can address.

Min Jun concludes by writing: "Training got the industry to build fast networks; agentic AI will determine which of those networks were built to last."

It should be noted that this is a firsthand description of CoreWeave’s own architecture. Claims regarding the scale of its operations, including having one of the largest InfiniBand and RoCE networks and one of the largest DPU deployments, are self-reported and not independently verified. The figure of 128,000 GPUs represents the design upper limit of the multi-plane topology; the article does not specify whether this is a deployed or planned configuration. All performance figures come from vendor self-testing, not third-party benchmarks.

Sources

  • wf.coreweave.com↗
Continue researching →