Scope Creep in AI Agents: CodeRabbit Founder Harjot Gill and OpenCode’s Ryan Vogel on Why Deciding What to Keep Is Harder Than Generation
When AI produces two hundred pull requests a day, even if all are reviewed, whether the changes are necessary remains an open question. Gill and Vogel explain how scope creep occurs, how reference points counter it, and under what conditions this approach is effective.
When AI agents can produce two hundred pull requests (PRs) a day or pile up six hundred pending changes for review at once, the core issue facing engineering teams is no longer whether the code has bugs, but whether these changes should exist at all. CodeRabbit founder Harjot Gill and OpenCode maintainer Ryan Vogel discussed this issue on The Merge podcast. Their consensus is that while AI-written code is useful, the harder part is deciding what to merge, what to push back on, and what to drop.
This judgment problem has an easily overlooked starting point: the problem takes shape before the code exists. Vogel gave the example of a kitchen to-do app—assign the task to an agent, and it will first add a database, then a caching layer, then more infrastructure. Each step individually does not look wrong, but the application ends up far more complex than a simple to-do list requires. Vogel pointed out that without an external reference point, every decision provides justification for the next, and the project can drift off course even while each step appears productive.
To combat this creep, Vogel and Gill each approached from different stages. Vogel’s method involves having the agent output an architecture plan first, which humans then challenge by identifying unnecessary parts; simultaneously, he runs a separate agent to compare intermediate outputs against the original requirement direction. Gill, drawing from CodeRabbit’s practice, added another dimension: review should not just look at diffs, but read changes in the order they were built. CodeRabbit divides review into three layers—the bottom layer consists of data and schema changes, the middle layer covers functions, business logic, and API routes, and the top layer includes the interface and its tests. Schema changes explain why an API behaves differently, and API changes explain what the interface now needs to handle. Reviewing in this order allows humans to check the assumptions connecting various parts, rather than guessing from filenames.