Amazon CEO Andy Jassy: Inference Costs Remain an Order of Magnitude Too High, Enterprise AI Adoption Stuck in the Middle Ground
In an a16z podcast interview, Amazon CEO Andy Jassy described AI demand as having a barbell distribution: one end consists of AI labs consuming massive compute power, and the other end comprises enterprises using AI for cost avoidance. The middle ground—enterprise production workloads that have not yet widely adopted inference—is currently stalled. Jassy argues that inference costs remain an order of magnitude higher than the level required for widespread enterprise adoption, identifying this as the core economic barrier preventing the activation of the middle segment.
Amazon CEO Andy Jassy used the 'barbell' metaphor to describe the current distribution of AI demand during an interview with the a16z (Andreessen Horowitz) podcast on October 23, 2026.
Jassy provided a specific mechanism for the stagnation in the middle ground: inference costs. He stated that although inference costs have decreased significantly over the past few years, they remain an order of magnitude more expensive than the level necessary for broad enterprise adoption.
The practical significance of this mechanism for readers is that it provides a trackable variable to gauge the progress of enterprise AI adoption. If inference costs continue to fall, the threshold for enterprises to rewrite their applications will lower, thereby truly activating the middle ground. Jassy mentioned that Amazon's self-developed Trainium chips offer approximately 30% to 40% better price-performance compared to similar competitors, and he views the self-developed chip business as a key component in driving down inference costs.
However, Jassy’s argument has clear boundaries. His reference to 'an order of magnitude' includes the personal qualifier 'in my opinion,' and he explicitly stated that he finds it difficult to predict the specific speed of enterprise adoption.
From the demand side, Jassy also revealed the extent of supply constraints at Amazon: Amazon expects to invest approximately $220 billion in capital expenditures in 2026. Despite this massive investment, he noted that significant demand remains unmet: capacity for 2026 is sold out, most of 2027 is sold out, and a substantial portion of 2028 is also sold out.