ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

Nvidia Packs Optical Components into Switch Chip Packages to Target Data Transfer Bottlenecks in AI Clusters

This in-depth report from Scientific American explains why Nvidia is replacing pluggable optical modules with a co-packaged optics solution: human operations in data centers are responsible for 5% of downtime, old optical modules consume about 25 watts each, and integrated optical engines are claimed to increase mean time between interruptions by at least 10 times. The article also reveals Nvidia's investment layout in the silicon photonics ecosystem and its strategic intent to control the communication layer of AI infrastructure.

Verified 2026-11-18 05:27 GMT+8

The bottleneck in large-scale AI clusters is shifting from computation to data transfer. Gilad Shainer, Senior Vice President of Networking at Nvidia, said in an interview with Scientific American that the rule is simple: use copper whenever you can. But copper is only suitable for short-distance, low-bandwidth scenarios. When tens or even hundreds of thousands of compute nodes and memory units in a data center need to communicate with each other at extremely high speeds, the problems with copper become apparent: electrons carry charge and repel each other, requiring power to drive them; copper itself acts like a filter, smearing clear electrical pulses into blurred noise.

Photons do not have this problem. They carry no charge and hardly interact with one another. A laser beam passing through glass fiber optic cables experiences negligible signal attenuation over several kilometers. However, data inside chips is in electronic form. To hand it off to photons for transmission and then back to electronics for processing, a conversion device is required—a transceiver.

The traditional approach is to make transceivers into pluggable modules inserted into the front panel of switches. Craig Thompson, Vice President of Interconnect Products at Nvidia, explained the fatal weakness of this method: physical connectors can loosen and move, having pins, solder joints, and epoxy, and every insertion or removal affects adjacent modules. Shainer provided a figure: the presence of humans in data centers is itself responsible for 5% of downtime.

Sign in / Sign up
ReadingResearchRadarInvestment framework

The idea behind Co-Packaged Optics (CPO) is to integrate the optical engine directly into the package of the switch chip. Nvidia’s latest switches—the Quantum-X and Spectrum-X—feature small photonic chips arranged on both sides of the central large chip, communicating face-to-face with electronic chips. Fiber optic cables plug directly into the front panel of the switch, eliminating the need for pluggable modules. Thompson stated that their modeling shows this can improve Mean Time Between Interruptions (MTBI) by at least 10 times.

Improvements in power consumption are equally specific. Old-style pluggable transceivers each carried their own lasers and required a dedicated chip to clean up electrical signals, consuming about 25 watts per unit together. A single switch holds dozens, and these modules can consume 10% of the total power in an entire data center. Placing the optical engine next to the chip means signals barely need to travel over copper, allowing the same electricity bill to buy more computing power.

From a strategic perspective, Nvidia designs approximately 85% of the graphics processors used for AI globally and has already sold most of them. If it simultaneously controls the way these chips communicate, competitors will find it difficult to replace Nvidia from AI infrastructure. The article also mentions that Nvidia has invested in laser manufacturers Coherent and Lumentum to help them expand capacity, and is collaborating with chip foundries to develop manufacturing processes for photonic circuits.

Competitors are not absent. Broadcom is already on its third generation of co-packaged switches, while AMD and Intel are also advancing similar technologies. However, Nvidia’s advantage lies in scale: the number of chips it sells is sufficient to support building the entire silicon photonics ecosystem it needs.

A further direction is optical interconnects within the chip itself. Processors are limited by the physical size of photolithography masks; a single chip cannot exceed approximately 850 square millimeters. To make them larger, multiple small chips must be stitched together. These small chips communicate via fine copper wires, and the denser the wires, the more severe the crosstalk. Sudip Shekhar, an electrical engineer at the University of British Columbia, believes photons may perform better than copper in these short-distance hops.

It should be noted that the data regarding the 10x MTBI improvement and power savings cited in the article are results of internal Nvidia modeling, not independently verified by third parties. The article’s author also mentioned that the Nvidia souvenir he received—a model of a 2026 server rack—still contained network interconnect components from before the era of co-packaged optics, indicating the rapid pace of this technology's iteration.

Sources

  • scientificamerican.comScientific American↗
Continue researching →