Prompt caching, read from your bill
How to audit what prompt caching saves: the two cache columns on every request, a hit rate you can compute, and the three patterns that quietly kill your savings.
Prompt caching is sold as a discount. Your bill is where that discount is either proven or silently lost. Most explanations stop at the mechanism; this post teaches you to read your own numbers — because a saving you can't verify is just a smaller number you're still trusting.
Disclosure up front: I operate Teapot Router, an API gateway whose workspace shows per-request cost, tokens, and cache columns with export. The audit method below works on any endpoint that exposes the same numbers.
Two columns decide your discount
Caching bills on two special token counts. Cache-creation tokens: the first write of a stable prompt prefix, billed at a modest premium over normal input price. Cache-read tokens: later reuses of that same prefix, billed at a deep discount — commonly an order of magnitude below cold input. Exact multiples differ per provider and model; if they are not on the published price list, that itself is a finding.
The mechanism is prefix reuse: whatever sits stably at the front of your prompt — system prompt, project context, earlier conversation — is processed once and then re-read cheaply on every later turn. The discount is not a coupon. It is the arithmetic consequence of the front of your prompt not changing.
The number worth computing: hit rate
Pull a week of real requests and compute one ratio: cache-read tokens over total input tokens. The absolute value depends on your workload — a fresh-chat-heavy user and a long-session agent will land in different places — so the trend is the signal, not a universal target. A hit rate that slides downward across a week usually means something upstream of pricing changed: your prompts, your session shape, or your endpoint.
Three patterns that quietly kill your cache
When the trend slides, look for one of these three before blaming the provider:
- An unstable prefix. A timestamp, a rotating instruction, or an edited system prompt at the very front invalidates the cached prefix every turn — you re-pay the write premium and never collect a read. The front of the prompt must be boring.
- Context churn in the middle. Agents that paste giant, ever-changing tool outputs between turns keep the prefix moving; depending on where that churn sits, reuse collapses. Clearing stale tool output is a caching decision as much as a tidiness one.
- Sessions shorter than the cache. Caches are short-lived — minutes, not days. Long idle gaps between turns reset the clock, and a session chopped into many small ones may never re-read enough to amortize the write premium. Fewer, longer, more stable sessions cache better.
The auditability question
Whatever endpoint you use, three questions decide whether the caching discount is real for you: Does the per-request ledger split cache-creation, cache-read, and cold tokens? Are cache prices in the machine-readable price list, not just a blog post? Can you export enough history to see your hit-rate trend? If any answer is no, the saving is a marketing claim you cannot verify — and unverified discounts have a way of evaporating.
The workspace ledger and public pricing endpoint are how we answer those questions on our own service: every request's cache columns, exportable, priced from the same JSON the calculator reads: https://teapotrouter.com/en/claude-code-cost-calculator
Prompt caching is not a feature you enable; it is a behavior you measure. Read the two columns, compute the hit rate, watch the trend, and fix the prefix before shopping for a better rate. A provider that shows you the columns is letting you check their work — one that doesn't is asking you to grade your own discount with your eyes closed.
FAQ
Does prompt caching always save money?
No. Cache writes carry a premium over normal input price, so a prefix that is used once costs slightly more. Savings come from stable prefixes re-read many times — long sessions, big static system prompts, reused documents.
How do I compute my cache hit rate?
From the per-request ledger: cache-read tokens divided by total input tokens over a week of real work. Judge the trend, not one request — a sliding hit rate points at prompt or session changes before it points at pricing.
Why did my caching savings disappear?
Usually one of three: the prompt prefix stopped being stable (edits or volatile content at the front), idle gaps grew longer than the cache lifetime, or the endpoint changed how it accounts for cache — which you can only see if the ledger splits the columns.