Modal CEO Erik Bernhardsson: Compute Contracts Sell Rectangles, Real Workloads Are Not
In a Latent Space podcast interview, Modal CEO Erik Bernhardsson explains why GPU compute contracts are sold as fixed rectangles, how real AI workloads fail to match this shape, and how Modal achieves duration transformation by aggregating thousands of customer demands. He also highlights the downside price risk inherent in three-year lock-in periods.
The biggest structural problem in the GPU compute market is not a shortage of chips, but that the shape of the contracts is wrong. Erik Bernhardsson, CEO of the AI infrastructure company Modal, offered a concise diagnosis on the Latent Space podcast: almost all GPU capacity is sold in the form of a three-year rectangle—fixed quantities of cards, fixed durations, with delivery starting three to six months from now. But real AI workloads are not rectangular.
He cited a specific figure: the median lifespan of sandboxes on the Modal platform is approximately eight seconds. A coding agent might create an environment, run code, check the results, and try again, repeating this cycle many times. Inference requests fluctuate throughout the day according to user activity cycles, while training tasks exhibit sudden spikes. These shapes do not match fixed-quantity, fixed-duration rectangular contracts at all.
Modal’s solution, which Bernhardsson calls "duration transformation," aggregates the needs of thousands of customers to smooth out the overall curve. It then locks in underlying capacity and sells it to each customer in the amount and time they actually need. The higher the degree of aggregation, the better the smoothing effect and the higher the utilization rate. He described this as one of the core pillars of Modal’s value proposition.
Whether this model works depends on startup speed. If new instances take minutes to become ready, elasticity is meaningless. To address this, Modal built its own file system, process checkpointing, and GPU memory snapshotting mechanisms, aiming to spin up a sandbox within milliseconds. Bernhardsson believes that only by reducing the friction of starting and stopping to a sufficiently low level can large-scale parallelism of "one environment per command" be achieved.