Reading

According to a report by The Decoder citing Axios, several major US banks, including Morgan Stanley, JPMorgan, Bank of America, and Wells Fargo, are restricting external AI agents from logging into bank accounts using customer-provided credentials. This practice violates bank policies that require account holders to log in personally.

Some AI agents have been observed acting autonomously without direct user instructions, raising significant security concerns. Morgan Stanley has already blocked AI agents entirely on its platforms, while JPMorgan, Bank of America, and Citi are implementing similar measures. Although most banks aim to limit agents to read-only access—allowing them to view data but not execute actions—many struggle to distinguish whether an agent is acting on behalf of a user or accessing an account independently.

Users typically grant AI access through browser tools like Computer Use or open-source agent software such as OpenClaw, making provider-side blocking technically challenging. Meanwhile, OpenAI has launched a dedicated ChatGPT service for financial tasks in the US with read-only access to financial data, indicating the industry's move toward compliant alternatives.

Extremely fast power fluctuations in AI data centers are posing a new threat to grid frequency stability. At the recent Data Center World Power conference, industry experts warned that AI workload oscillations occur in millisecond increments, timeframes the traditional grid is not designed to track or counter quickly.

The modern grid was designed for predictability and stability. Morning and evening peaks are forecastable, giving grid operators time to bring additional generation online and then ramp it down. However, this cadence is incompatible with AI workloads, which can spike and subside in seconds or even milliseconds. Extreme events, such as a sudden drop of hundreds of megawatts in AI power draw, could trigger wide-area disturbances if not mitigated.

Sai Somayajula, principal electrical design engineer at Nvidia, noted that power density within the rack is increasing with every generation as AI evolves. From 5-10 kW a few years ago, to about 40 kW in the H100 era, roughly 150 kW for Blackwell systems, and targeting around 240 kW for Vera Rubin, future projections reach up to 1 MW per rack. To blunt the impact of rapid load swings, Nvidia has added rack-level storage and controls to smooth fast excursions within the rack and keep the AC input within grid requirements.

Rajesh Gopanath, core infrastructure engineering architect at Oracle Cloud, warned about subsynchronous oscillations. These frequencies below the US grid’s 60 Hz operating frequency, if excited by AI data centers’ behavior, can induce mechanical resonance leading to catastrophic failure, including damage to turbine shafts. He emphasized that no single technology can solve all these problems, requiring capacitors, batteries, uninterruptible power supplies, low-voltage ride-through, power smoothing, and fast-responding generation.

David Roop, who leads the power systems engineering division at Mitsubishi Electric Power Products, stated that supporting AI workloads requires coordinated investments at the substation and grid levels. Success depends on a chip-to-grid approach that accounts for short- and long-duration energy storage, harmonic performance, power quality, and local grid characteristics. New guidelines, standards, and compliance criteria are being introduced to deal with instability much faster.

According to a report by TechNews, New York-based AI infrastructure startup Fragment announced the completion of a $12 million funding round, co-led by FirstMark and BoxGroup. The company focuses on providing a security control layer for AI agents executing financial operations, addressing growing enterprise demand as "agentic commerce" rises.

Fragment’s core technology embeds “double-entry bookkeeping” logic into software to ensure transaction balances before funds move. This mechanism aims to fully record the source and destination of funds, thereby reducing the risk of significant financial losses caused by AI agent hallucinations, errors, or bypassing safety limits. Previously, Fragment had completed an $11 million seed round led by Stripe.

The company currently serves approximately 50 to 60 paying clients, including corporate credit card provider Ramp, financial software company BILL, and digital marketplace Whop. Although the valuation was not disclosed, notable investors and existing customers indicate that the market is beginning to prioritize security controls for AI-automated payments.

US-based AI company Anthropic announced on October 22, 2026, that it is donating its Agent Skills standard to the Agentic AI Foundation (AAIF), a project under the Linux Foundation. The foundation was co-founded last year by Anthropic, Block, and OpenAI, and previously received the Model Context Protocol (MCP) specification.

In an official statement, Anthropic said: "We’re committed to keeping Agent Skills open, community-driven, and vendor-neutral." This move means the standard will no longer be controlled by a single vendor but managed by a neutral body, aiming to reduce developer dependency on specific AI models and promote interoperability of agent skills across different platforms.

Prominent developer Simon Willison noted that this development aligns with his prediction from December last year, when he suggested Anthropic would likely donate the Skills standard to the AAIF. With both MCP and Agent Skills now housed in the same foundation, the underlying communication and capability extension mechanisms for AI agents are accelerating toward open standards.

Google announced Agent Compute Engine on October 22, 2026, a service designed to provide each AI agent with its own hardware-isolated micro virtual machine (microVM). The service currently powers Gemini internally and will open for preview in the coming months.

According to Google, these microVMs are secured by the company's Titanium chip, enabling hardware-level isolation. When an agent idles, its VM automatically pauses to free up resources for others. Shared memory between VMs significantly reduces the cost of running individual agents. Google claims that in test environments, this technology allows a single server to host ten times as many agents. In a test involving 200,000 agents, suspend and resume operations took less than 500 milliseconds.

This efficiency addresses the high overhead traditionally associated with VMs in agentic systems. As AI agents scale, resource isolation and scheduling have become critical infrastructure bottlenecks. Google also open-sourced Agent Substrate, a Kubernetes runtime supporting this architecture. On the governance side, Google added token tracking, model routing, and support for third-party identity providers like Ping Identity and SailPoint, extending these controls to Cloud Run, GKE, and managed endpoints.

Most teams can get an agent to complete a task once. Getting the same agent to complete that task on the 100th identical request, at a cost you can forecast, with a trace you can read when it fails, is a different engineering problem. Weights & Biases by CoreWeave blog author Shreyan Basu Ray attributed the core mechanism of this gap in a long-form piece published in July 2026 to one sentence: the cost driver is the loop resending accumulated context each iteration.

Specifically, a ten-step agent does not make ten small calls. It makes ten increasingly large ones. System prompts, tool schemas, conversation history, and every prior tool result are resent in each iteration. Data reported by Anthropic gives the shape: agent interactions use roughly 4x the tokens of chat interactions, and multi-agent systems roughly 15x. Agentic workloads routinely run input-to-output token ratios of 100:1 or worse, which means agentic cost is an input-token problem, not an output-token problem—intuitions built on chat pricing are off by an order of magnitude.

The article illustrates the mechanism with a concrete example of a customer service ticket triage agent: assume a 6,000-token system prompt with tool schemas, 8 steps, roughly 1,500 tokens of tool results and reasoning added per step, and 300 output tokens per step. A single run totals about 90,000 input tokens and 2,400 output tokens. At representative frontier pricing ($3 per million input tokens, $15 per million output tokens), that is approximately $0.31 per task. Compare that to a single classification call using the same system prompt, which costs about $0.02. The agent is roughly 14 times more expensive, and the entire difference comes from context resent across turns.

If the stable system prompt is cached (with a typical cache-read rate of one-tenth of base input), the same run drops to about $0.18, a reduction of about 42%. At a scale of 100,000 tasks per month, this single change is worth about $13,000. A systematic evaluation across three major providers on a multi-turn research benchmark found that prompt caching reduced API costs by 41% to 80%, improved time to first token by 13% to 31%, and savings scaled with prompt size, reaching nearly 90% at 50,000-token prompts.

But the article emphasizes a counterintuitive finding: naive full-context caching is not the optimal strategy. Caching only the stable system prompt was most consistent across both cost and latency, because full-context caching triggers cache writes for dynamic tool calls and results that will never be reused, adding overhead without corresponding reads. On one model, full-context caching produced a latency regression of about 9%, while system-prompt-only caching improved latency by about 31%. Practical rules are therefore very specific: keep the cacheable prefix stable, do not place timestamps, session identifiers, or user-specific strings at the beginning of the system prompt, because one differing token at the start invalidates the entire prefix.

Architecture choices impact cost far more than model selection. The article notes that the strongest published evidence for architecture-driven savings is model-tier orchestration: expensive models plan and delegate, cheap models execute. Anthropic reported in its multi-agent documentation that on the BrowseComp benchmark, an orchestrator model delegating to cheaper worker sub-agents reached 96% of the all-frontier configuration's performance at 46% of the cost—specifically, 86.8% accuracy at $18.53 per problem versus 90.8% accuracy at $40.56 per problem. The mechanism is simple: the orchestrator only sees plans and summaries, while workers absorb the bulk of the token-heavy reading and writing at the lower rate.

This saving holds under two conditions: workers must not genuinely need frontier reasoning for their subtasks, or you pay twice through retries; and repeat calls should land on the same worker so each keeps its own warm cache instead of re-paying full price for shared context.

A more subtle issue than cost is the honest measurement of success rates. τ-bench introduced pass^k—the fraction of k independent attempts at the same task that all succeed. The results are sobering: an agent resolving over 60% of retail tasks on a single attempt dropped to about 25% when required to succeed across eight attempts. A system that looks like a 60% performer is closer to one in four measured honestly. If your agent faces real users, the hundredth identical request matters more than the first.

The MAST Multi-Agent System Failure Taxonomy, based on analysis of over 1,600 annotated execution traces across seven frameworks, identified 14 failure modes. About 40% belong to specification and system design issues, about a third to inter-agent alignment failures, and the rest to verification failures. The authors explicitly note that improvements in base model capability are insufficient to cover the entire taxonomy—waiting for the next generation of models to fix reliability suggests a long wait, because the defects lie in the system design itself.

This cost structure directly rewrites priorities for inference infrastructure. In workloads with an input-to-output ratio of about 100:1, the majority of GPU time is spent processing inputs rather than generating outputs—the bottleneck is context, not generation. KV cache hit rate thus elevates from a tuning detail to a first-class operational metric, because a high hit rate directly eliminates the dominant cost.

The optimization order given by the article is: fix context first, then caching, then routing, and finally consider training, measuring after each step. Skipping intermediate steps to switch to a larger model usually reveals that you paid more to solve a problem that belonged in the context layer.

Gartner predicted in June 2025 that over 40% of agentic AI projects would be canceled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls. Model capability is not on that list—the failures are economic and managerial, not capability-based.

Cost per call flatters agents. If a triage agent has a 70% success rate and retries once upon failure, the effective cost per successful task is much higher than the per-run number, because failed runs consume the full token budget before failing. Benchmarks or dashboards that ignore failed runs always make agents look cheaper than they are.

The same effect appears in published benchmark data: on the SWE-bench verified subset, multiple configurations reached identical 54.0% accuracy, but costs ranged from about $259 to about $1,790—same accuracy, roughly seven times the spend. Without a cost axis, leaderboards cannot tell you which to deploy.

There is a documented cliff when budgets tighten. In TestEvo-Bench, capping per-task spend at $1 dropped one agent's success rate on test generation from 70.6% to 44.2%, and another from 59.5% to 18.8%. Budget caps are a real control mechanism, but they are not free; different agents degrade in very different ways under budget pressure.

Token spend is just the visible part. Production budgets also bear tool and API call costs, retrieval and vector database infrastructure, trace storage and observability, evaluation overhead, human escalation handling—which is often the largest true unit cost when an agent routes to a person—and the engineering time to maintain tool contracts.

The tail matters more than the mean. Runs generated by agents that recursively spawn subagents or hit retry storms can cost orders of magnitude more than the median, and without per-run caps those runs are unbounded. Only by tagging cost per task in traces from day one and classifying by outcome can you calculate cost per successful task, cost per escalation, and the cost distribution.

The article simultaneously draws boundaries: when deterministic rules, SQL queries, classifiers, search indexes, or RPA scripts can reliably solve the task, agents are the wrong tool. Agents add latency, cost, nondeterminism, and operational surface area, requiring genuine uncertainty or adaptation problems to justify them. Choosing a deterministic alternative is not a step down; it is matching the tool to the problem.

There is an instructive divergence in the multi-agent field. Anthropic reported that its multi-agent research system outperformed single-agent configurations by 90.2% on internal evaluations, while also reporting a roughly 15x token multiplier and noting that token usage explained most of the performance variance. Cognition held a nearly opposite view in "Don't Build Multi-Agents": agents acting on partial context make conflicting decisions, so a single thread produces more coherent results. Both are right about different task shapes—the former is breadth-first research where subtasks are truly independent; the latter is coding, where each decision constrains the next.

The advice running through all choices is: choose the least autonomous architecture that satisfies the task, then add agentic flexibility only where the workflow genuinely needs runtime decisions. Architecture determines cost; task shape determines architecture.

The question in production environments is not whether an agent can complete a task once, but whether the workflow can fail safely, improve continuously under evaluation, and stay within latency and cost budgets. Every item in the current design consensus—from explicit termination conditions to typed tools to trajectory evaluation—exists because someone has already paid the tuition for not doing so. Compare against a deterministic baseline before scaling: if the agent cannot beat the baseline on measurable dimensions, that is also a useful result, and the cost learned now is far smaller than after launch.

New Enterprise Associates (NEA), an American venture capital firm, partner Philip Chopin and Hunter Worland opened their explanation for why they are leading Spiko’s $90 million Series B financing round for the Paris and London-based corporate cash management platform with a single sentence: too many businesses pay to lose money. More precisely, they pay management fees, performance fees, and transaction fees to hold cash that loses purchasing power every year on an inflation-adjusted basis.

The problem they identified is not that any specific bank offers low interest rates, but that the underlying infrastructure for corporate cash management remains stuck in the 20th century. Each layer is operated by a different institution, uses different ledgers, reconciles via overnight batch files, and exposes data to customers through outdated interfaces. The result is that businesses collectively lose billions of euros in yield annually, simply because the pipes beneath their cash have not been rebuilt.

Building on this diagnosis, Chopin and Worland define tokenization as the next foundational shift in financial infrastructure. Its mechanism involves migrating assets onto shared ledgers where ownership and balances exist on a single programmable ledger, enabling instant settlement, continuous yield accumulation, and software-driven workflows to replace batch processing and manual operations.

They further propose a forward-looking connection: this same structure serves as the natural foundation for AI agents that will increasingly run financial functions. On legacy bank and transfer agent infrastructure, automating finance is difficult because every action requires reconciling multiple siloed ledgers and waiting for overnight files; on a tokenized stack, AI agents can execute smart contracts, automatically sweep surplus cash into yield, post collateral, rebalance exposures within guardrails, and turn today’s static financial balances into always-on, self-optimizing systems.

Regarding why the approach starts in Europe, the authors cite three conditions: non-financial enterprises earn significantly lower yields than global peers, banks have underinvested in technology, and Spiko has already assembled a complete regulatory licensing stack. They simultaneously argue that the opportunity is fundamentally global, with the same architecture applicable to the United States, Latin America, and Asia.

Boundaries for readers to note: NEA is the lead investor in this round, and this text constitutes a public statement of its investment thesis. The cited figures of approximately $2.7 billion in assets under management and over 10,000 customers come from company self-reports as of September 2026, viewable on the public Dune dashboard, though NEA explicitly states in its disclaimer that certain information comes from third-party sources and has not been independently verified by them. The estimate of billions of euros in lost yield is provided by the authors without methodological detail. The assertion that AI agents run financial functions is a forward-looking judgment by the authors, not an achieved fact. Whether tokenized cash funds can evolve from supplementary tools for European corporate cash into mainstream infrastructure depends on the regulatory environment, institutional adoption speed, and operational frictions, which are not fully elaborated in this argument.

OmniParseBench, released by Datalab in 2026, is an open document parsing benchmark. It covers 2,937 pages of documents—including scanned forms, handwriting, arXiv mathematical formulas, and dense tables—using 16,288 pass/fail tests. It published item-level verdicts for 27 parsers, allowing anyone to download these verdict files and recalculate the leaderboard.

A developer did not simply cite the leaderboard numbers but downloaded the published verdict files and recalculated all items using the benchmark's own scoring function. The recalculation confirmed that all 108 numerical values on the leaderboard card are reproducible, but it also exposed three notable issues.

The first issue: the top two parsers are actually tied. Datalab accurate and Gemini 3.8 Flash both have a headline score rounded to 92.9, with a difference of only 0.024 percentage points. The developer used document-level bootstrap—sampling 2,343 documents with replacement 2,000 times and recalculating the headline each time—to estimate ranking uncertainty. The results showed that the 95% confidence interval for the gap between the two ranges from approximately -0.9 to +0.9, crossing zero. In each sample, each led roughly half the time. The honest interpretation is that they are indistinguishable.

Why use document-level rather than test-level bootstrap? Because tests on the same document tend to pass or fail together; independent sampling of individual tests would underestimate uncertainty. This choice is a key step in the audit methodology.

The second issue: rankings flip depending on document type. The developer recalculated headlines by slicing into 13 categories including handwriting, tiny fonts, rotated text, degraded scans, math, tables, forms, multi-column layouts, and scripts such as Greek, Chinese, Cyrillic, Armenian, and Lao. The overall leader, Datalab accurate, remained ahead in only 3 of these slices. Gemini 3.8 Flash led in handwriting, math, and forms; Claude Sonnet 5.5 led on rotated pages; and Mistral OCR led in Greek. The Armenian and Lao slices contained only 6 and 5 documents respectively, which the developer explicitly treated as hints rather than rankings.

The third issue: Datalab's own published figures are inconsistent with the files. The launch article and blog post listed Datalab accurate at 92.7 and ranked it behind Gemini, whereas the actual value in the files showed a difference of only 0.024 percentage points—a tie that rounds to the same figure. The developer verified the launch statements item by item; 13 of 17 checks held, while 4 did not (including Datalab's two ranking figures, the reproduction difference for Indra OCR, and the direction of the placebo test).

Another interesting finding was reported as a flag rather than proof. The OmniParseBench dataset card notes that Gemini 3.8 Flash and Claude Sonnet 5.5 participated in the creation and verification of some tests. The developer used a difference-in-differences test to ask: do these two star models perform abnormally well on the 102 tests corrected by vision models compared to other tests? The result showed that on the model_review slice, the pass rate drop for both models was about 25 percentage points less than for other parsers, while the human review placebo slice showed a smaller gap in the same direction, and the model_panel slice showed the opposite direction. The developer explicitly stated this is merely a flag and does not constitute proof.

Practical lessons: For any benchmark leaderboard, first confirm whether you can access the item-level verdict data; if so, use document-level resampling to estimate uncertainty rather than looking at a single headline number; recalculate rankings by slicing according to the document types you actually care about; verify whether vendor-reported figures match public files; and lock version numbers and dataset commits, as repositories change as new parsers are added.

The most common latency experience issue for voice agents is not that model generation is too slow, but that synthesis starts too late. A user asks a question, the language model is already generating a response token by token, but the text-to-speech module waits until the entire response is complete before it begins working. A single-sentence response might be generated in 600 milliseconds, but a full multi-paragraph response takes five seconds, and the integration logic waits for those five seconds. During these five seconds, the user hears nothing.

A technical solution released by the AWS SageMaker AI team on October 22, 2026, points out that the key to solving this problem is not running synthesis and generation in parallel, but changing the timing of synthesis startup: as soon as the language model produces a complete sentence, send that sentence immediately to the TTS model, without waiting for subsequent content. They deployed Kokoro-82M (with only 82 million parameters) on a SageMaker AI real-time endpoint, maintaining a persistent connection via a bidirectional WebSocket stream, sending text sentence by sentence and receiving audio sentence by sentence.

There is a potential misunderstanding here. The team explicitly clarifies that this is not concurrent synthesis—the loop of the bidirectional processor is strictly sequential: receive a message, synthesize its audio, send the audio, send a completion signal, then receive the next message. The audio for the second sentence does not begin generating while the first sentence is still being transmitted. What incremental sending actually achieves is that synthesis for the first sentence begins the instant it exists, rather than waiting for the full response. For multi-sentence responses, there is a genuine generation time gap between the first and last sentences; this gap represents the entire latency benefit.

Isolated measurements show that sending only the first sentence (rather than the full four-sentence text) reduced the time to first audio byte at the TTS transport layer from 329.3 milliseconds to 156.7 milliseconds, a relative improvement of 52.4%. In end-to-end comparisons, the bidirectional streaming mode was 2.4 times faster than batch mode (local container) and 3.5 times faster (deployed AgentCore runtime). The gap actually widens when real network hops are added, because the streaming mode absorbs network overhead within the overlap window of generation and synthesis, whereas every serial wait step in batch mode directly exposes network latency.

There is a second cost at the transport layer. If the frontend is already streaming text via Server-Sent Events (SSE), the team measured the cost of stuffing base64-encoded audio into SSE events: payload inflation is approximately 33%, and the SSE delimiter (empty line) does not align with internal MP3 frame boundaries—in practice, four logical MP3 frames arrived as six transmission blocks, two of which were truncated mid-frame. They compared MP3 at 48kbps and AAC 32kbps fMP4 encoding, confirming with Amazon Transcribe that decoding quality showed no difference.

For projects with an existing SSE pipeline, the base64 approach is a pragmatic choice, at the cost of 33% extra payload and a buffer reassembly step. For scenarios building a voice path from scratch or prioritizing latency, the team recommends directly reusing binary audio and JSON control frames on the same bidirectional session—the distinguishing rule is: if a data block cannot be parsed as a valid JSON object, it is audio. This rule works because although raw PCM audio may start with the byte 0x7b (left curly brace), the entire PCM frame will almost never coincidentally constitute a valid JSON object.

All data comes from self-tests conducted by the AWS team on their own SageMaker AI endpoints and Bedrock AgentCore runtime; independent third-party reproduction was not provided. The 82-million-parameter scale of Kokoro-82M is suitable for running on a single mid-tier GPU instance, but the article does not provide subjective quality assessments of synthesized speech or quality comparisons with other TTS models. The example container pins an older version of PyTorch with a known security advisory (PYSEC-2026-2286); the team notes that an upgrade is required before formal use.

The biggest structural problem in the GPU compute market is not a shortage of chips, but that the shape of the contracts is wrong. Erik Bernhardsson, CEO of the AI infrastructure company Modal, offered a concise diagnosis on the Latent Space podcast: almost all GPU capacity is sold in the form of a three-year rectangle—fixed quantities of cards, fixed durations, with delivery starting three to six months from now. But real AI workloads are not rectangular.

He cited a specific figure: the median lifespan of sandboxes on the Modal platform is approximately eight seconds. A coding agent might create an environment, run code, check the results, and try again, repeating this cycle many times. Inference requests fluctuate throughout the day according to user activity cycles, while training tasks exhibit sudden spikes. These shapes do not match fixed-quantity, fixed-duration rectangular contracts at all.

Modal’s solution, which Bernhardsson calls "duration transformation," aggregates the needs of thousands of customers to smooth out the overall curve. It then locks in underlying capacity and sells it to each customer in the amount and time they actually need. The higher the degree of aggregation, the better the smoothing effect and the higher the utilization rate. He described this as one of the core pillars of Modal’s value proposition.

Whether this model works depends on startup speed. If new instances take minutes to become ready, elasticity is meaningless. To address this, Modal built its own file system, process checkpointing, and GPU memory snapshotting mechanisms, aiming to spin up a sandbox within milliseconds. Bernhardsson believes that only by reducing the friction of starting and stopping to a sufficiently low level can large-scale parallelism of "one environment per command" be achieved.

However, the other side of the coin is financial risk. To secure the capacity needed for 2027, Modal has had to sign contracts exceeding three years with nearly 25 cloud service providers and prepay significant funds. If GPU prices fall in the future—which Bernhardsson believes is the long-term trend for hardware prices—companies locking in high-priced capacity will be left holding expensive supply. While he does not expect a crash in the near term, he acknowledges this is a possibility that must be considered when building the business.

This model carries an implicit scale prerequisite: the number of customers must be large enough and their demand correlations low enough for the aggregated curve to truly smooth out. Bernhardsson did not provide a specific customer count required to reach this threshold, nor did he explain how the mechanism degrades if demand becomes highly correlated. This represents the boundary conditions of duration transformation as a business logic.

According to a report by Semafor, cited by TechCrunch, Elon Musk has invested approximately $1 billion in Skydance, the newly formed media company. The news emerges weeks after Paramount completed its $110 billion acquisition of Warner Bros. Discovery to create the combined entity.

The report notes that Musk's investment coincided with equity contributions from other Skydance investors, including board member Laurene Powell Jobs. This capital provides a much-needed financial boost as the company seeks to cut costs by $6 billion over the next three years and lift its stock price.

Skydance is co-led by CEOs David Ellison and Ynon Kreiz, tasked with integrating streaming platforms Paramount+ and HBO Max alongside networks like CBS and CNN. Musk has close ties with Larry Ellison, whom he appointed to Tesla's board in 2018; Larry Ellison also invested $1 billion to help Musk purchase Twitter in 2022.

Skydance did not immediately respond to requests for comment. The specific terms, valuation basis, and final closing status remain unconfirmed by the issuer, presenting this strictly as a reported fact.

When training large models, networks face a rhythmic problem: thousands of GPUs simultaneously execute the same collective communication, over and over, for weeks. Bandwidth pressure is immense, but the pattern is predictable; engineers can design around this rhythm.

In a blog post published in August 2026, Min Jun of CoreWeave, an AI cloud infrastructure company, described another type of traffic: agents break tasks into steps, call retrievers, call tools, call another model, sometimes call another agent, and wait for each step to return before knowing what the next one is. This serial path is an agent chain, and its latency is the sum of every hop, not the worst single hop.

This distinction seems minor, yet it changes the logic of entire network design. When thousands of such chains run on the same hardware alongside training tasks and standard inference, traffic no longer resembles batch jobs; it becomes a ubiquitous, latency-sensitive, multi-hop conversation. Assumptions never tested by training begin to break.

The first consequence Min Jun explains is that oversubscription is not an option. When leaf switches have less uplink capacity to the spine than the nodes below them can generate, this gap remains invisible until two types of traffic compete for the same link. While training alone can tolerate some degree of oversubscription, a network mixing synchronized training bursts with thousands of small, sequential agent requests must simultaneously satisfy high aggregate bandwidth and low, predictable per-flow latency; oversubscribed links cannot achieve both. CoreWeave’s approach is to maintain a 1:1 non-blocking ratio at every layer.

The second consequence involves how the network scales. Traditional two-tier leaf-spine topologies eventually exhaust switch radix limits; the solution is adding a third tier, but this increases hop count—which is precisely what agent chains cannot afford. CoreWeave adopts a multi-plane topology: distributing each GPU’s SuperNIC connections across two or more independent network planes, trading depth for width. Min Jun states this allows a flat, two-tier design to scale to 128,000 GPUs, 64 times the coverage of a single-plane network, without requiring redesign.

The third consequence is that load balancing must move from software to hardware. Traditionally handled by NCCL at the software layer to disperse collective communication traffic across available links, this approach carries two costs: competing with host CPUs for cycles, and only reacting to congestion after the fact rather than proactively routing around it as conditions change. CoreWeave uses NVIDIA ConnectX-9 SuperNICs to offload this task to hardware, allowing dedicated hardware paths to respond to real-time congestion telemetry at line rate. In its NVIDIA Vera Rubin NVL72 deployment, each GPU’s SuperNIC runs at up to 1,600 Gb/s; switches coordinate directly with SuperNICs, dispersing traffic across every available path, plane, and rail using real-time telemetry rather than static hashing.

The fourth consequence is that resilience cannot be centralized. A training job stopping for a few minutes is expensive, but it can recover from checkpoints. An agent stuck halfway through a ten-step chain, multiplied across thousands of concurrent chains, has no equivalent recovery point. Min Jun describes building resilience across five layers: the topology layer shifts traffic off degraded paths using multi-plane and multi-rail designs; the silicon layer uses BlueField-3 DPUs to place networking, isolation, and policy enforcement on dedicated hardware, freeing host CPU cycles; the software layer uses CoreWeave Kubernetes Service (CKS) to completely remove the hypervisor, ensuring virtualization faults do not become tenant issues; the operations layer uses Mission Control to automatically replace nodes once they cross health thresholds; and the geographic layer links multiple data centers into a single backbone, ensuring cross-site workloads do not depend on any single facility.

The fifth consequence is that observability must reach every hop. If the slowest hop is hidden, all previous design efforts are futile. CoreWeave uses NVIDIA network observability tools to trace flow-level performance from GPU to SuperNIC, transforming the vague symptom of “this agent chain feels slow” into specific links or queues that engineers can address.

Min Jun concludes by writing: "Training got the industry to build fast networks; agentic AI will determine which of those networks were built to last."

It should be noted that this is a firsthand description of CoreWeave’s own architecture. Claims regarding the scale of its operations, including having one of the largest InfiniBand and RoCE networks and one of the largest DPU deployments, are self-reported and not independently verified. The figure of 128,000 GPUs represents the design upper limit of the multi-plane topology; the article does not specify whether this is a deployed or planned configuration. All performance figures come from vendor self-testing, not third-party benchmarks.