Cloudflare engineer Ryan Skidmore: after AI code review costs doubled, the biggest single saving came from fixing cache prefix sharing
The average cost of Cloudflare's internal AI code review rose from $1.19 to $2.46 with no architectural changes. Engineer Ryan Skidmore discovered that although the cache hit rate reached 97%, costs still doubled — the root cause was that each session's first request hit the cache only 14% to 26% of the time, because the system prompt placed shared and repository-specific content in the same block with the only cache breakpoint at the end of that block, preventing the shared portion from being cached independently. After fixing the prefix structure, cost dropped to $0.77, but the quality-validation methods themselves have limitations.
重要度重大証拠E3 検査可能執筆簡易
Cloudflare runs an internal AI code review system: a coordinator dispatches merge requests to specialist reviewers for code quality, security, performance and other dimensions, each independently calling a large language model to complete its analysis. In April 2026, an average review cost $1.19; by the end of August, that figure had climbed to about $2.46. No significant architectural changes had been made — the coordinator still handed work to the same set of specialist reviewers, and the same risk tiers determined review intensity.
Engineer Ryan Skidmore documented the entire cost-optimization process on Cloudflare's engineering blog. He first ruled out an intuitive explanation: it wasn't that review volume had increased. The chart holds the mix of review tiers fixed, so a day with more large reviews doesn't skew the data. Cost did steadily rise from $1.22 in March to $2.32 in August, with a small part attributable to more reviews landing in the most expensive tier (59% in March, then 65% in August) — as models grew more capable, engineers produced bigger, more complex changes, naturally pushing more into the 'full review' tier.
But most of the cost increase came from model changes. At the start of August, every reviewer migrated onto Anthropic models, raising cost per job from $1.38 to $2.03. The internal standards reviewer, Codex, moved to Claude Opus 4.7, taking its per-review cost from about $0.25 to $0.88 — more expensive than the coordinator itself. Meanwhile, the cache hit rate rose from 85.7% in April to 97% by September, while tokens per review more than tripled from 0.91 million to 2.99 million. A high hit rate on three times as many tokens still costs a lot more, even with the low cache-read price.
The real diagnostic began on September 14. The system started recording the token usage of each reviewer's first LLM call, plus a hash of the first system block exactly as sent to the provider. The data revealed a counterintuitive fact: although every session starts with the same tool definitions and the same agent prompt (between 11k and 21k tokens, identical across every repository), the first request hit the cached prefix only 14% to 26% of the time.
The problem lay in how OpenCode v1 constructed the system prompt. It built one block containing the agent prompt, followed by an environment section with the working directory and the date, and placed the only cache breakpoint at the end of that block. This meant the shared part was never stored on its own — a review in one repository could never reuse what a reviewer in another repository had written. Anthropic's prompt cache works by caching prefixes: only the contiguous unchanged messages at the start of a conversation are eligible; content after a breakpoint that differs each time cannot be hit.
The September 16 fix was straightforward: add a marker to the end of every agent prompt and use a plugin hook to split the original single block at that marker. The first half now ended on bytes that every job shares and got its own breakpoint. To keep it that way, anything repository-specific had to be removed from in front of it — the list of available skills was disabled and tool descriptions were fixed across jobs. The same day, Codex's reference index (a table of internal standards, a skill file and a list of reference files) was moved directly into its prompt, eliminating the first three turns of tool calls that every session previously spent reading those files. Codex's LLM calls per run dropped from 15.9 to 9.6, and its cost per run from $1.12 to $0.72.
Together these changes took cost per review from $2.47 down to $2.02. A batch of smaller changes followed on September 21 and 23: removing tools that sub-reviewers couldn't use (in 57% to 76% of sessions they tried to call the comment-posting tool, were rejected by a server check, and wasted another call recovering); letting reviewers read files in parallel instead of one at a time; skipping reviewers on re-reviews who had nothing new to examine; stopping after the review was posted instead of generating an unseen summary; treating a slow reviewer differently from a broken provider; and cancelling jobs that were already cancelled (GitLab's Kubernetes executor doesn't send a signal when a job is cancelled — in a sample of 446 cancelled jobs, 358 kept running until the review finished). None were impressive individually, but together they saved about $0.20 per review.
The most involved change was sharing one prefix between three reviewers. The code quality, security, and performance reviewers now send identical first requests — the same tools, a shared rules file, then the merge request with every patch included — with each reviewer's own instructions following after. The system ensures one writes the cache and the other two read from it. But a cache entry can't be read until the response that wrote it has started streaming, and all three start simultaneously, so the second and third wait up to 20 seconds for the first response to begin before sending their own request. This slightly slows them down but saves about nine cents per reviewer.
Model and effort-level adjustments contributed the larger share of savings. The author split the period from early September through October at every release and configuration change, finding that code changes accounted for $0.65 of the $1.67 total saving, while model and effort changes accounted for $1.06. The coordinator had been running three model and effort combinations since August; on the same model, full-tier jobs at medium effort finished 20% faster, so the other two were dropped. A later randomized split showed Opus 5.5 at medium produced no measurable change in review outcomes or critical findings compared with Opus 5, so they switched. Opus 5.5 is cheaper per token than Opus 5, but price only explains about $0.20 of the drop; the rest comes from sending and generating fewer tokens.
On October 3, every Sonnet reviewer was split 50/50 between Sonnet 5 and Sonnet 5.5. After two and a half days, Sonnet 5.5 was 62% cheaper per run for code quality, 58% for performance and 47% for security — the two models cost the same per token, so all savings came from using fewer tokens. On full-tier reviews, the 5.5 reviewers posted about as many critical and warning findings as the Sonnet 5 ones, so on October 5 all were moved to 5.5. At medium effort, the code quality reviewer now makes 1.7 LLM calls per run instead of 6.2, and more than half of its runs finish in a single call, meaning they don't open any other file in the repository.
But does lower cost mean lower quality? Skidmore tested this two ways. First: for samples of critical and warning findings, compare the reviewed commit with the author's next push, counting a finding as acted on if the lines it pointed at changed. Lines 30 to 40 away from each finding changed about 15% of the time in every period, providing a baseline for coincidental edits. Before the changes, authors acted on 34% of the code quality reviewer's serious findings; now it's 59%. The reviewer flags fewer things, but those it flags are much more likely to lead to a change, so findings that led to a change per 1,000 reviews stayed about the same: 132 now versus 145 before, within the margin of error.
Second: two models from different providers judged a blinded sample of 657 findings from the three Sonnet reviewers, with the diff each one referred to and without knowing which period it came from; disagreements went to a third model. Each finding was labelled important, minor, duplicate, wrong, or unverifiable, and its severity marked as fair, too high, or too low. Most of what disappeared was minor findings, plus some duplicates and findings the judges couldn't verify from the diff. Important findings per review went from 0.28 before to 0.30 now, with overlapping intervals. The bigger change was in severity: before, judges rated 0.46 critical or warning findings per review as more severe than they deserved; now that's 0.14, a 70% drop and the clearest result in the research.
But the answer depends on the reviewer. For code quality at high effort, what disappeared was mostly noise and inflated severity, and it still finds about as many real problems as before. Medium effort did lose real findings: in the randomized split, code quality at medium led to roughly a quarter as many acted-on findings as at high. Performance also lost real findings: judges found about 64% fewer important performance issues per review. Security fell by about a quarter, though that isn't statistically significant. The author explicitly notes: 'this isn't a statistically sound controlled experiment' — the judges are models too, each group has only 35 to 66 findings, and the 'now' window is only a day long.
In the end, average review cost dropped from $2.43 to $0.77, and median review time from 4 minutes 9 seconds to 1 minute 32 seconds. Skidmore identifies the biggest single saving as checking what actually reached the provider on each session's first request, and finding a way to order those messages as consistently as possible across commits, merge requests and repositories. His two recommendations for anyone running agents at volume: first check whether each session's first request actually reads from the cache, then check whether your reviewers still find real problems at the effort level you're paying for.