AI Data Company Snorkel Says Elastic Bursting Mechanism Reduced Agent Simulation Sandbox Costs by Approximately 20%
Snorkel's agent simulations face dual challenges of resource usage spikes and unpredictable workloads, where static configurations lead to failures or waste. The company adopted Modal's elastic bursting mechanism to scale resources on demand; the Senior Manager of AI Platform stated that total sandbox bills were thereby reduced by approximately 20%, with about 10-15% of sandboxes currently enabled for this feature.
Resource configuration for agent simulations presents a fundamental contradiction: all code within the sandbox is generated by large language models, making the execution pattern impossible to determine in advance. Throughout a sandbox's lifecycle, it often experiences periods of silence while waiting for model responses, immediately followed by instantaneous spikes such as grep commands or file traversals. The height of these spikes depends on exactly what code the model generates, which is unknowable at the time of configuration.
AI data company Snorkel previously estimated CPU and memory quotas for each sandbox, resulting in a lose-lose situation: tasks would crash midway due to insufficient quotas, invalidating the entire run; conversely, abundant resources led to idleness and inefficient spending. Rustem Feyzkhanov, Snorkel's Senior Manager of AI Platform, stated that they needed to support various tasks from domain experts and agents without over-provisioning.
Modal's elastic bursting mechanism changed this landscape. Sandboxes specify a baseline resource amount during configuration but can elastically expand CPU or memory on demand during runtime. Billing rules take the higher of actual usage or requested amount. This means Snorkel can lower initial requests—because bursting capabilities cover the spikes—without needing to reserve margins for worst-case scenarios. Feyzkhanov stated that this change reduced their total sandbox bill by approximately 20%.