AI Civic Data Company Hamlet: Production Data Branching Lets Coding Agents Independently Verify Each PR
Hamlet Lead Developer Erich Douglass and Senior AI Engineer Jeff Hui describe a concrete change: after giving each pull request its own branch of production data, AI coding agents can run data pipelines to completion and check their own results, while a human still approves the merge. Previously, preparing test data consumed several hours per engineer each week.
Hamlet is an AI-powered civic data platform that indexes and transcribes city council, planning commission, and zoning board meetings across the United States, allowing real estate developers, journalists, researchers, and citizens to search meeting content and track local policy developments. CEO Sunil Rajaraman says customers want to know whether a council member will approve their housing project, why or why not, and what history the city has that might make it hesitate. This business requires processing extremely messy data from more than 7,200 governing bodies, and the engineering team is small—velocity matters more than almost anything.
Lead Developer Erich Douglass recalled the previous state: they ran one production instance and one staging instance on AWS RDS, with much of their work involving tuning data pipelines against very specific data. Every pull request required 15 to 30 minutes to get staging into the right state before they could run validations. Senior AI Engineer Jeff Hui added that 15 to 30 minutes was the best case—when you happened to copy exactly the right data. Each engineer could spend several hours a week keeping test data current, including cloning pieces of production locally to chase down issues.
After switching to Databricks Lakebase Postgres, every pull request automatically gets its own branch of production data, connected to its preview environment. Douglass says when the preview deploys, the data is ready—they can test against production data and even write to it without touching production. Hui says now when they hit a bug, they have the exact data that caused it, and they can check the fix against that same data.