Amazon CEO Jassy: AI Demand Is Barbell-Shaped; Inference Costs Determine Whether the Middle Segment Opens Up
In an a16z podcast, Amazon CEO Andy Jassy stated that current AI demand is barbell-shaped: one end consists of AI labs and a few hit applications consuming massive compute power, while the other end comprises enterprises using AI for cost avoidance. The middle segment—existing enterprise production workloads—is largely not yet widely adopting inference. He argues that inference costs remain an order of magnitude higher than what is needed for widespread adoption, and that self-developed chips (Trainium), which offer a 30% to 40% improvement in price-performance, are a key lever in lowering this barrier.
In an October 23, 2026 episode of the a16z podcast, Amazon CEO Andy Jassy described a demand structure he has repeatedly observed: AI demand is distributed in a barbell shape. One end of the barbell features AI labs and a few runaway applications (such as ChatGPT and Claude Code) that consume unprecedented amounts of compute power; the other end features numerous enterprises using AI for cost avoidance and productivity gains, such as automating customer service, fraud detection, and business process optimization. The middle segment of the barbell, which involves integrating existing enterprise production workloads with inference, represents the largest absolute market opportunity in his view, but the vast majority have not yet truly begun.
This observation itself is not new—many cloud providers have said similar things. Jassy’s contribution lies in linking a specific cost mechanism to this structure: why is the middle segment stuck? He believes inference costs are still an order of magnitude more expensive than the level required for widespread adoption. His reasoning is that if enterprises need to rewrite existing CPU-based applications to use AI accelerators, and the cost gap between the two solutions is large, the rewrite must yield very high incremental returns to be worthwhile; however, if inference costs come down, this threshold drops significantly, making almost all applications better, smarter, and more capable with AI accelerators and inference.