In an October 23, 2026 episode of the a16z podcast, Amazon CEO Andy Jassy described a demand structure he has repeatedly observed: AI demand is distributed in a barbell shape. One end of the barbell features AI labs and a few runaway applications (such as ChatGPT and Claude Code) that consume unprecedented amounts of compute power; the other end features numerous enterprises using AI for cost avoidance and productivity gains, such as automating customer service, fraud detection, and business process optimization. The middle segment of the barbell, which involves integrating existing enterprise production workloads with inference, represents the largest absolute market opportunity in his view, but the vast majority have not yet truly begun.
This observation itself is not new—many cloud providers have said similar things. Jassy’s contribution lies in linking a specific cost mechanism to this structure: why is the middle segment stuck? He believes inference costs are still an order of magnitude more expensive than the level required for widespread adoption. His reasoning is that if enterprises need to rewrite existing CPU-based applications to use AI accelerators, and the cost gap between the two solutions is large, the rewrite must yield very high incremental returns to be worthwhile; however, if inference costs come down, this threshold drops significantly, making almost all applications better, smarter, and more capable with AI accelerators and inference.
He then embeds Amazon’s chip strategy into this logic. Annapurna Labs was not originally acquired for AI chips—the core motivation for the 2015 acquisition was to offload virtualization, networking, security, and storage overhead from CPUs to specialized network interface cards, allowing servers to dedicate as much compute power as possible to customers. The team was subsequently used to build Graviton CPUs (which he claims are about 40% more price-performant than x86 processors, used by 99% of Top 1000 EC2 customers), and later Trainium AI accelerators (which he claims are 30% to 40% more price-performant than comparable processors). His judgment is that self-developed chips are the primary lever for reducing inference costs and thereby opening up the middle segment.
Boundaries to retain: These figures are estimates made by Jassy and self-reported data from Amazon, without independent verification. 'An order of magnitude' is his personal judgment, not a measured result. Reasons for the middle segment's lack of AI adoption may extend beyond cost—Jassy also cited talent familiarity with post-training and fine-tuning, and organizational ability to prioritize, as influencing factors. Jassy himself admitted it is 'hard to predict the rate at which this will happen.' Additionally, a16z, as a venture capital firm, has interests in Amazon and the companies discussed; while this does not invalidate Jassy’s observations, readers should be aware this is not neutral third-party analysis.
Nevertheless, the value of this framework for readers lies in breaking down the vague judgment that 'AI adoption is slow' into a trackable structure: both ends of the barbell are already growing (lab compute consumption, enterprise cost optimization), the middle segment is the future's largest increment, and inference costs are the observable variable connecting them. When pricing and performance data for Trainium or similar self-developed chips become public, readers can test whether this mechanism is being realized.