- Cloudflare says its AI code review cost fell 68%, to $0.77 per review, after the average had roughly doubled from April to August with no redesign.
- Before the work, Cloudflare says cache writes were 43% of review spend and cache reads 34%. Model and effort changes gave $1.06 of a $1.67 saving, code changes $0.65.
- Ask your teams to report AI cost per task, split by output, cache writes and cache reads, and to name who approves model swaps.
An AI bill can double while the design stays the same
The price per token tells you little about what an AI system will cost to run. Cloudflare's own figures show why. In April, its AI code reviewer averaged $1.19 per review. By the end of August, the average was about $2.46.
Cloudflare says the design stayed the same over that period: one coordinator and the same specialist reviewers. In mid-September it asked its own AI agents to trace where the tokens were going and trim them. A review with the code quality reviewer at high effort now averages $0.77. Measured against $2.43, the average for its two priciest weeks, that is a 68% fall.
Cloudflare traces most of the earlier rise to model changes. A shift toward its costliest review tier, from 59% of reviews in March to 65% in August, explains a small part. The rise followed decisions made for sound reasons, such as better results on the workload. The lesson here is that AI cost is an operating discipline, not a one-time purchase decision.
Where the money went: reloading context
An AI model has no memory between calls, so each call sends the whole conversation again. Cloudflare's code quality reviewer made 16.6 calls per review on average. Across them it wrote 72k tokens to a cache and read back 1.08 million. Tokens are the chunks of text a model processes and bills by.
A cache saves the start of a conversation so the model need not process it again. That saved start is called a prefix. Under Anthropic's terms, as Cloudflare lists them, saving a prefix costs a quarter more than ordinary input, and reusing it costs a tenth. A saved prefix lapses after five idle minutes.
Before the cost work, Cloudflare says output was about 23% of review spend, cache writes 43% and cache reads 34%. Output was under a quarter of the bill. Handling cached context took the rest. Cloudflare puts the size of an average review at 0.91 million tokens in April and 2.99 million by September, more than three times as large. The cache hit rate rose from 85.7% to 97% over the same stretch. A high hit rate on three times the tokens still costs more.
A cache that kept missing
From September 14, Cloudflare logged the first call of every session. On that opening call, a reusable cached prefix existed in just 14% to 26% of cases. That was odd. The first 11k to 21k tokens, tool definitions and agent instructions, are the same in every repository Cloudflare reviews.
The cause sat in OpenCode v1, the older agent software. It assembled the system prompt as a single block: the agent instructions first, then an environment section with details such as the current folder and date. The only cache marker came after all of it. The shared instructions were therefore never saved on their own, and a review in one repository could not benefit from work done in another.
Picture a shared company handbook stapled to a cover note that names one customer and the date. The bundle is sealed, so it can only be filed as a whole, and no two bundles match. Nothing gets reused. Separate the handbook from the cover note and the handbook can be filed once and used again.
That is close to what Cloudflare did on September 16. It added a marker that splits the prompt in two and gave the shared half its own cache point. The same day it moved its standards reviewer's reference tables into the prompt, cutting that reviewer's calls from 15.9 to 9.6 per run. Once both changes shipped, the average review fell from $2.47 to $2.02.
A later change made three reviewers share one prefix. The shared entry becomes usable only once the first reviewer's response begins to arrive. The other two hold back for up to 20 seconds before their first call. Cloudflare says the wait saves about nine cents per reviewer.
Model and effort choices moved more than code
Cloudflare splits the $1.67 saving into $0.65 from code changes and $1.06 from model and effort changes. The cache fix was one of the code changes. Cloudflare cautions that each step compares weekday hours before and after a change, so this is not a controlled experiment.
The same levers had pushed costs up. In early August, Cloudflare moved all its reviewers onto Anthropic models, which its evaluations favoured for this workload. The cost of a job climbed from $1.38 to $2.03. Codex, its internal standards reviewer, runs on Claude Opus 4.7. Its cost per review rose from about $0.25 to $0.88 in August, more than the coordinator agent.
The latest change was a model swap. On October 3, Cloudflare ran Sonnet 5 and Sonnet 5.5 side by side on half the traffic each. Over two and a half days, the newer model cost 62% less per run for code quality. Cloudflare says both models carry the same token price, so the saving reflects less text processed, not a lower rate. At medium effort, the code quality reviewer made 1.7 calls per run instead of 6.2.
Cheaper, but did it find less?
Fewer serious flags is the obvious worry. Cloudflare's code quality reviewer, now on Sonnet 5.5 at high effort, raises 40% fewer critical and warning findings than in late August. Cloudflare's security and performance reviewers, which stay at medium effort, post 56% and 71% fewer. Cloudflare says a drop alone cannot show whether real problems slip through or noise has been cut.
So it checked whether the lines a finding pointed at changed in the author's next push. For code quality, the share of serious findings followed by an edit rose from 34% to 59%. Lines 30 to 40 away from a finding changed about 15% of the time, which gives a coincidence baseline. In absolute terms, acted-on findings per 1,000 code quality reviews barely moved: 145 before, 132 now. Cloudflare calls that gap within the margin of error. Across all four reviewers combined, they rose from 229 to 321.
This check was reported for code quality and for all four reviewers together, not for security and performance separately. These are one company's measurements of one internal workload. An edited line is a proxy for a real problem, not proof of one.
What to ask your teams
First, ask for cost per task, not per token, split into output, cache writes and cache reads. In Cloudflare's case, output was the smaller share.
Second, ask who approves a model or effort change, and whether anyone measures its cost. Cloudflare's August move was made for performance reasons and raised the bill.
Third, ask whether the shared start of every prompt is stored on its own, apart from job-specific details such as folder and date. Cloudflare's cache missed until it split them.
Fourth, ask how quality is measured beyond a count of findings. Cloudflare checked whether authors acted on them.
An AI agent's bill follows the tokens it reads and writes. The useful question is how many of them the task truly needs.
Produced by the WebPulse Newsroom with AI assistance from the original reporting credited below, and checked against that source by our editorial review. How we use AI.
Original reporting: Cloudflare.
