Databricks Application Scaling Guide: Separating Soft and Hard State to Decide Between Adding Machines or Instances
Bruno Tafur and Tushar Madan of Databricks published an extension guide for application builders in November 2026, discussing how to maintain fast response times and high availability as a Databricks App grows from a few users to hundreds or thousands. The core judgment of the article is that the intuitive answer 'add more compute resources' is only effective once you know where requests spend their time and whether the application itself is built to scale.
The authors divide the scaling process into four stages: design, build, test deployment, and operations. In the architecture design stage, the most critical decision is dividing application state into two categories. Soft state is temporary working data, such as in-memory context, caches, and ongoing work, which can be rebuilt from hard state if lost; hard state consists of business records, checkpoints, and user preferences, which must be persisted in Lakebase or Unity Catalog tables so that any instance can recover the session. This classification determines whether horizontal scaling is truly feasible—if session state exists only in memory, users will lose their work when instances restart or migrate.
The article provides a specific retailer scenario to demonstrate load estimation: during the busiest hour of Black Friday, 200 out of 2,000 store managers are using an inventory query app simultaneously, with each performing two queries per minute, each executing three SQL statements. The result is 1,200 statements per minute, averaging 20 per second. The authors emphasize that this estimates query rate rather than concurrency; testing needs to validate burst traffic and growth, and database connections should be budgeted separately.
Regarding connection budgeting, the article provides an actionable formula: when each worker maintains a connection pool, the upper limit of connections is the number of instances multiplied by the number of workers per instance multiplied by the maximum pool size. For example, with five instances, eight workers each, and a pool limit of ten, up to 400 connections can be opened. After adding other clients and connections opened on behalf of users, this total must be compared against database documentation limits, leaving headroom for deployments. For scenarios requiring queries to Lakebase on behalf of users, each user requires an independent Postgres role, and Lakebase limits branches to 500 roles; adding instances does not change this upper limit.
The core recommendation for the build phase is to use asynchronous code. Most applications spend time waiting for external services—SQL queries, model calls, and external APIs. Blocking calls occupy an execution slot for the duration of the wait, meaning a few slow calls can exhaust capacity. When handlers await model and database calls, one worker can continue serving other requests while waiting. The article also recommends using asyncio.gather or Promise.all to launch independent queries in parallel, making request latency close to that of the slowest query rather than the sum of all queries, while limiting parallelism to avoid overwhelming the backend.
Sources:Databricks