Meta Runs Google TPUs in Production for First Time, Challenging Nvidia's Dominance

According to Tom's Hardware, Meta is now running Google's Tensor Processing Units (TPUs) in production within its data centers. This marks the first time Google's custom AI chips have been used by an external major customer for actual business workloads. The breakthrough is enabled by a newly released PyTorch-native version of the SGLang inference engine, developed through a close collaboration between Meta, Google, and RadixArk, the startup founded by SGLang's creators.

Previously, TPUs could only be accessed via Google's proprietary JAX framework or the older PyTorch/XLA bridge, limiting their adoption in the mainstream PyTorch ecosystem. The new TorchTPU engine allows developers to deploy PyTorch workloads directly to TPUs without rewriting code. Zongwei Zhou, Meta's PyTorch-TPU lead, stated that the engine provides Meta with "a remarkably mature, PyTorch-native serving stack."

This development poses a potential threat to Nvidia, which has built a strong moat around its CUDA software ecosystem. Meta, as a primary supporter of the PyTorch framework, shifting to TPUs signals that large tech companies are actively seeking to break free from single-hardware constraints. Jana van Greunen, Director of PyTorch Engineering at Meta, noted that Meta "cannot be constrained by a single hardware architecture" and that TorchTPU enables them to deploy PyTorch workloads to Google TPUs.

It is worth noting that this is not Meta's first attempt to use TPUs. Google describes PyTorch/XLA as a "predecessor" to the new TorchTPU, and SemiAnalysis reported last year that Meta research teams used TPUs through this bridge from 2020 to 2023 before canceling those contracts. The current deployment is based on a multi-year rental agreement, though specific scale and locations remain undisclosed.