Modal reports Nvidia Vera CPU doubles throughput in internal tests, plans heterogeneous fleet

AI cloud provider Modal has reported that preliminary internal evaluations showed Nvidia's Vera CPUs delivered twice the throughput of its existing AMD and Intel processors for specific agentic AI workloads. Despite this performance advantage, Modal CEO Erik Bernhardsson confirmed that the company does not intend to cease using x86 CPUs entirely. Instead, Modal plans to operate a heterogeneous server fleet, selecting the most suitable processors for each workload to balance performance against software migration costs.

This decision highlights the growing importance of CPUs in agentic AI, where numerous independent tasks compete for cores, cache, and memory bandwidth. However, because Vera is based on the Arm v9.2 instruction set architecture, adopting it requires substantial modifications to Modal's software stack. The company has not disclosed detailed benchmark configurations, specific AMD or Intel processor models, or performance-per-watt figures. Consequently, it remains unclear whether the advantage stems from the Olympus CPU cores' decoding capabilities, the 1.2 TB/s memory subsystem, or other architectural features. Notably, the tests compared Vera against currently available competitors rather than upcoming generations.