EstontincPublished 3 min read
How to reduce AI costs without losing quality: lessons from Cloudflare's 68% cut
Cloudflare's AI code reviewer doubled in cost without any big change, then dropped 68% after optimisation. Where the money went, what fixed it, and how to apply the same lessons.
How do you reduce AI costs? Many teams discover that their AI bill grows faster than their usage. On 19 October 2026, Cloudflare described exactly that happening to its internal AI code reviewer, and how it brought the average cost per review down by 68%, from $2.43 to $0.77, while checking that quality held up.
This article explains why AI costs creep up, what Cloudflare changed, and a checklist you can apply to any AI feature. If you are still planning a feature, start with our guide on estimating AI API costs.
Why AI costs creep up
Cloudflare's average review cost rose from $1.22 in March to $2.32 in August without significant architectural changes. Its explanation: models are getting better and more thorough, so agents read more context and make more calls. Tokens per review more than tripled, from 0.91 million to 2.99 million.
Before optimising, most of the money went on context the model had already seen or was about to see again: cache writes were about 43% of spend and cache reads about 34%, with output tokens around 23%. Each call sends the whole conversation, so cost grows with both the number of calls and the size of the context.
What Cloudflare changed
- Shared cached prefix. The first request of a session found a cached prefix only 14% to 26% of the time, although the tool definitions and agent prompt were identical. Restructuring the prompt so the shared part could be cached on its own was the biggest single saving.
- Fewer calls. Moving a reference index into the prompt cut one reviewer's calls per run from 15.9 to 9.6. Hiding a tool sub-reviewers should not use removed a wasted call in most sessions.
- One prefix for three reviewers. The code quality, security and performance reviewers now send identical first requests, so later ones read from the cache.
- Model and effort chosen by testing. Randomised splits showed which model and effort level kept quality while using fewer tokens.
About 40% of the savings came from code changes and the rest from model and effort choices. Since early September, cache reads per review fell 82%, cache writes 54% and output 67%. The slowest reviews also improved: reviews taking more than five minutes fell from 37.9% to 0.7%.
Did quality suffer?
Cloudflare did not assume quality held. It checked whether authors acted on flagged findings, and had models from two providers judge a blinded sample of 657 findings. For code quality at high effort, what disappeared was mostly noise: authors acted on 59% of serious findings, up from 34%. At medium effort, some real findings were lost, so effort levels were tested per reviewer rather than cut across the board.
AI cost checklist
Based on Cloudflare's results and our own project experience, these habits keep AI features affordable:
- Measure tokens per task, split into input, cache reads, cache writes and output.
- Keep shared prompts identical and at the start of every request, so they can be read from cache.
- Remove steps nobody reads, such as summaries or tool calls whose results are never used.
- Give agents only the tools they need, so they do not waste calls on tools they cannot use.
- Test models and effort levels with real tasks, not assumptions. Our guide to choosing an AI model for your business covers the trade-offs.
- Check quality after every change, with a sample of real outputs, before rolling it out.
What it means for your business
AI features are not a one-time cost. As models improve and usage grows, spending can double without anyone noticing. Treating token spend like server spend, measured, reviewed and optimised regularly, keeps AI projects profitable without sacrificing results.
How Estontinc helps
Estontinc is a software house in Malang with 100+ projects and 50+ clients over more than six years. Through our AI development service we design prompts, caching and model choices for AI features, and test that quality holds. Our cloud and DevOps team sets up monitoring so token spend stays visible over time.
Is your AI bill still climbing? Tell us about the features you run through the contact page, and we will reply within 24 hours.
Frequently asked questions
Why do AI costs keep going up?
As models get more capable, agents tend to read more context and make more calls. At Cloudflare, tokens per review more than tripled between April and September. Because each call sends the whole conversation, cost grows with both the number of calls and the context size.
How did Cloudflare cut its AI costs by 68%?
Cloudflare restructured prompts so shared parts were read from cache, removed unnecessary calls and tools, shared one prefix across three reviewers, and chose models and effort levels through randomised tests. About 40% of savings came from code changes and the rest from model and effort.
Does reducing AI costs lower quality?
Not necessarily, but it must be measured. Cloudflare found that for code quality at high effort, mostly noise disappeared and authors acted on more findings. At medium effort, some real findings were lost, so each change should be tested against real outputs before rollout.
What is prompt caching?
Prompt caching lets a provider reuse the parts of a prompt it has already processed, usually at a lower price. It works on prefixes, the content at the start of a request, so keeping shared instructions and tools identical and first in every request makes caching effective.
Sources
Facts and figures in this article come from these official sources.
- AI
- AI development
- Cloudflare
- Cost
