A detached mechanical keyboard key marked "E" lies on a wooden desk beside a dark keyboard

ArticleaiDeep read

Prompt Caching: How It Actually Cuts Your LLM API Bill

Ahmad JAug 8, 20263 minUpdated Sep 14, 2026

Prompt caching isn't caching the model's answers: it's the provider reusing your prompt's processed prefix, so repeated context (system prompts, tools, documents, chat history) is billed at a fraction and skips re-processing. How it works, what it costs, and where it breaks.

A deep read: the full picture, with the receipts.

Key takeaways

Prompt caching saves the processed state of your input prefix so the model skips re-reading repeated content, cutting cached input token costs by roughly 90 percent on providers like Anthropic and DeepSeek.
The cached prefix must be byte-for-byte identical from the start of the prompt, meaning any change near the front, such as a live timestamp or reshuffled JSON, busts the cache for everything after it.
Every provider enforces a minimum prefix size before caching activates, ranging from about 1,024 tokens on OpenAI to up to 4,096 tokens on certain Anthropic models, and misses are silent with no error returned.
Cache writes on Anthropic cost a small premium over the standard input rate (1.25x for the 5-minute window, 2x for the 1-hour window), so the savings only materialize when the same prefix is reused multiple times before expiry.

Signalstrong3independent sources

If you're building on top of large language models through an API, prompt caching is one of the highest-leverage cost levers you have, and one of the most misunderstood. It does not mean storing the model's answers and handing back the same text next time. That's response caching, a different technique. Prompt caching is a feature of the model provider: it saves the processed state of the input you send, so that when your next request starts with the same content, the model skips re-reading it and you're billed a fraction of the price for that part. You still get a fresh, freshly generated answer every time. What you stop paying for is the model re-processing the same long preamble over and over.

That distinction matters, because the savings are large and the technique is nearly free to adopt, if you understand what actually gets cached.

	Prompt caching	Response caching
What it stores	The processed state of the input you send	The model's finished answer
What you get back	A fresh answer, generated every time	The same text as last time
What you stop paying for	The model re-reading the same long preamble	Generating the answer at all
Can it serve something stale?	No	Yes, and that is its main risk

How prompt caching works#

When a model receives your prompt, the first thing it does is read it, the "prefill" step, building an internal representation of every token before it writes a single word of output. For a short question that's cheap. For a prompt carrying a big system message, a set of tool definitions, a 40-page document, or a long conversation history, prefill is most of the work and most of the cost.

Here's the key fact: that internal representation is built left to right, and each token's state depends on every token before it. So the model can reuse its work only from the start of the prompt up to the first point where anything changes. Think of it like a bookmark in a book the model otherwise re-reads from page one on every call. Prompt caching drops a bookmark after the last unchanged page; next time, the model resumes from the bookmark instead of starting over. But the bookmark is only valid if every earlier page is word-for-word identical: change one token near the front and the bookmark is worthless from that point on.

This is why prompt caching is a prefix cache. The reused content has to sit at the beginning of your prompt and be byte-for-byte the same across requests. The variable part, the user's actual question, goes at the end.

Providers expose this in one of two ways:

How it is exposed	Who does it	What you have to do
Explicit	Anthropic Claude	Mark where the stable prefix ends with a `cache_control` breakpoint in the request. You decide what to cache, and you can set several breakpoints for layered content
Automatic	OpenAI, and Google Gemini's implicit mode	Nothing in code. The provider detects a repeated prefix and caches it for you. You just have to keep the front of your prompt stable

Either way there's a floor and a clock. Caching only starts above a minimum prefix size, and that floor is provider- and model-specific: below it, shorter prefixes silently don't cache. On Anthropic the minimum depends on the model: roughly 2,048 tokens on Sonnet 4.6 and about 4,096 tokens on Opus 4.8 and Haiku 4.5. OpenAI's automatic caching starts around 1,024 tokens. And the cache is short-lived: Anthropic's default lifetime is about 5 minutes, refreshed each time you hit it, with an opt-in 1-hour window; OpenAI and Google keep prefixes warm on their own schedules. Miss the window and the next request rebuilds the cache from scratch.

Provider and model	Minimum prefix before anything caches
Anthropic — Claude Sonnet 4.6	About 2,048 tokens
Anthropic — Claude Opus 4.8 and Haiku 4.5	About 4,096 tokens
OpenAI — automatic caching	About 1,024 tokens

Below the floor nothing caches and nothing tells you so.

What it costs, and what it saves#

The economics have two sides. A cache read (hit) is cheap: on Anthropic a cached input token bills at roughly one-tenth of the normal input rate, about a 90% discount. A cache write (miss) can cost a small premium: the first request that lays down the cache pays for those input tokens in full, and on Anthropic a bit more, 1.25× the input rate for the 5-minute cache and 2× for the 1-hour cache. Automatic caches (OpenAI) generally don't surcharge the write.

Claude model	Input, per million tokens	Cached input, per million tokens
Sonnet 4.6	$3	About $0.30
Haiku 4.5	$1	About $0.10
Opus 4.8	$5	About $0.50

Those are Anthropic's published list rates as of mid-2026, and the cost calculator runs them against your own volume. OpenAI and DeepSeek land in the same territory, DeepSeek's cache hits also being about a tenth of list, and Google's context caching applies a comparable discount.

Two consequences fall out of that. First, prompt caching pays off through reuse: you eat a slightly higher write once, then bank the discount on every hit before the cache expires. Cache a prefix you only send once and you've spent more, not less. Second, the bigger and more repetitive your stable context, the bigger the win, which is exactly the shape of the workloads people run today:

Agents and tool use: a long system prompt plus a fat block of tool definitions gets re-sent on every step of every run. Cache it once per session.
Multi-turn chat: the conversation history grows each turn, but its earlier turns don't change, so the whole prefix caches and only the newest message is new.
RAG and document Q&A: pin one large document in the prefix and ask twenty questions against it; the document is billed at the cache rate for nineteen of them.
Coding assistants: the same repository context rides along with every request.

And it's not only cheaper. Skipping prefill on cached tokens cuts latency too: time-to-first-token drops sharply when the model doesn't have to re-read tens of thousands of tokens before it starts answering.

Where it breaks#

Prompt caching is unusually easy to think you're using while getting nothing back.

The failure	What it looks like	What to do
A moving prefix	A live timestamp, a request ID or a "current date" line at the top of the system prompt, so every call looks new	Keep the front stable and push volatile content to the end
Traffic too sparse for the TTL	A low-traffic endpoint expires its cache between requests and pays the write premium each time	Caching rewards steady or bursty reuse, not the occasional lone call
A prefix below the minimum	Nothing caches, and the miss is silent: `cache_creation_input_tokens` stays 0, no error	Clear the provider's floor before expecting a discount
Scope and invalidation	The cache is keyed to your account, the exact model, and certain request settings	Expect a switch of model or a change to cached tool definitions to invalidate it

The failure modes are specific, and each one is worth reading in full:

A moving prefix. Anything that changes near the front of the prompt busts the cache for everything after it. The classic own-goal is injecting a live timestamp, a request ID, or a "current date" line at the top of the system prompt: every call looks new. Nondeterministic ordering (JSON keys, or retrieved chunks in a different order each time) does the same. Keep the front stable; push volatile content to the end.
Traffic too sparse for the TTL. With Anthropic's ~5-minute default lifetime, a low-traffic endpoint can expire its cache between requests and pay the write premium each time with nothing to amortize it against. Other providers keep prefixes warm longer, but the principle holds: caching rewards steady or bursty reuse, not the occasional lone call.
Prefixes below the minimum. Every provider has a floor below which nothing caches, and on Anthropic it's model-dependent, roughly 2,048 tokens on Sonnet 4.6 and around 4,096 on Opus 4.8 and Haiku 4.5, versus about 1,024 for OpenAI's automatic caching. Below the floor there's nothing to cache and small prompts don't benefit, and the miss is silent: cache_creation_input_tokens stays 0, no error.
Scope and invalidation. The cache is keyed to your account (and, increasingly, your workspace) and to the exact model and certain request settings: it isn't shared across accounts, and switching models or changing cached tool definitions invalidates it. That isolation is good for privacy; it also means you can't borrow someone else's warm cache.

One worry that does not apply here: staleness of the answer. Because the model still generates fresh output on every call, prompt caching can't serve you an out-of-date response the way response caching can. The "is the cached data still current?" problem belongs to caching the answer, not to caching the prompt.

Getting the most out of it#

The lever	What to do
Order the prompt static-to-dynamic	Stable, reusable material first — system prompt, tools, documents, examples — and the user's variable input last. This is the single biggest lever, and structure beats everything else here
Use layered breakpoints where you can	Content changes at different rates: tool definitions rarely, the system prompt occasionally, retrieved documents per session, the user turn every time. Where a provider allows several breakpoints, Anthropic supports up to four, place them at those boundaries so a change in one layer doesn't invalidate the layers above it
Keep prefixes byte-identical	No timestamps, random IDs or reshuffled JSON near the front. Serialize the stable part deterministically
Reuse inside the window	Fire related requests against the same context while it's warm, and consider the 1-hour TTL when reuse is predictable enough to justify the higher write cost
Measure the hit rate	Don't assume, read it. Anthropic returns `cache_creation_input_tokens` and `cache_read_input_tokens`; OpenAI reports `cached_tokens` in usage. If reads aren't dwarfing writes, your prefix is moving somewhere you didn't expect
Stack it with response caching when it fits	For requests that are truly identical end to end, caching the final answer is cheaper still than regenerating it. The two techniques solve different problems and combine cleanly

The bottom line#

Prompt caching is close to free to turn on, and for the long, repetitive context that agents, chat, RAG, and coding tools all run on, it takes a real bite out of the bill: up to about 90% off the cached portion of your input, plus a latency win from skipping the re-read. The catch is entirely in the details it rewards: a stable, front-loaded prefix, big enough to clear the provider's minimum, reused often enough to beat the clock. Get the prompt structure right and the discount is automatic. Get it wrong (a timestamp at the top, a prompt reshuffled every call), and you'll pay the write premium forever and wonder why the cache never hits.