Prompt Caching Explained: How to Cut Your LLM API Bill by Up to 90%

Last Updated: July 2026 | 11 min read

Quick Answer: Prompt caching is an LLM feature that stores the model's computed state for a repeated prompt prefix so it doesn't reprocess those tokens on the next request. When a new call starts with the same prefix — a long system prompt, your tool definitions, or a reused document — the model loads the saved state instead of recomputing it. Cached input is billed at roughly 10% of the normal input price, so prompt caching cuts input costs by up to 90% on repeated calls and usually lowers latency too. The output is identical; only the prefill step is skipped.

If your LLM bill is climbing faster than your usage, prompt caching is probably the single highest-leverage fix you're not using. Most teams resend the exact same 5,000-token system prompt on every request and pay full price for it every single time — even though the model already did that work seconds ago.

Prompt caching stops that waste. It's supported by every major provider in 2026 — Anthropic, OpenAI, and Google Gemini — and turning it on ranges from zero code changes to a single extra field. Yet it's the most under-used cost lever in production AI, mostly because the rules for making it actually work are poorly understood.

This guide explains exactly how prompt caching works, the real cost math (with break-even numbers), how the three big providers differ, and the silent mistakes that quietly make your cache do nothing. At SolutionGigs, applying the checklist at the end has cut client inference bills by more than half without touching a single line of prompt content.

What Is Prompt Caching?

Prompt caching is a technique that saves the model's internal computation for a repeated prompt prefix, so identical leading tokens are loaded from a cache instead of being reprocessed on every request.

When an LLM reads your prompt, it runs every token through its attention mechanism to build a set of key-value (KV) tensors — this is the prefill phase, and its cost scales roughly with prompt length. Prompt caching stores those tensors server-side. If your next request begins with the same bytes, the provider reuses the stored KV state and only processes the new tokens at the end.

The result: you pay full price for the fixed prefix once, then a heavily discounted "cache read" price on every request that reuses it. Nothing about the model, the weights, or the output changes — this is purely a billing and speed optimization. That makes it different from RAG or fine-tuning, which change what the model knows or how it behaves.

Prompt caching is ideal for: - Long, fixed system prompts reused across many requests - Tool/function definitions in agent loops - A large document or codebase queried with many different questions - Few-shot examples that stay constant - Multi-turn chat, where each turn reuses the whole prior conversation

How Does Prompt Caching Work?

Prompt caching is a prefix match: the provider hashes the exact bytes of your prompt up to a cache checkpoint, and any change anywhere before that point invalidates everything after it.

The rendered order of a request is tools → system → messages. Providers cache from the beginning of that sequence forward. So the golden rule is simple: put stable content first, volatile content last.

Here's the request lifecycle:

1. First request — the model runs prefill on the whole prompt. The fixed prefix is written to the cache (you pay a normal or slightly higher "write" price for those tokens).
2. Cache hit — a later request with the identical prefix loads the stored state. You pay the discounted "read" price for the cached tokens, plus full price for only the new tokens at the end.
3. Cache miss — if any byte in the prefix changed, or the cache expired, the model reprocesses from the point of difference and writes a fresh cache entry.

The three numbers to watch in every API response tell you exactly what happened:

• cache_creation_input_tokens — tokens written to the cache (write price)
• cache_read_input_tokens — tokens served from cache (~10% of input price)
• input_tokens — tokens processed fresh at full price

If cache_read_input_tokens stays at zero across repeated calls, your cache isn't working — jump to the cache-miss checklist below.

The Real Cost Math (With Break-Even Numbers)

Prompt caching pays off after just 2–3 requests that reuse the same prefix — and the more reads per write, the closer your savings get to the full 90%.

Let's make it concrete with Anthropic's Claude Opus 4.8 pricing ($5 per million input tokens, as of mid-2026):

Now imagine a 10,000-token system prompt sent with 100 different user questions.

• Without caching: 100 × 10,000 tokens × $5/1M = $5.00 just for the repeated prefix.
• With caching: 1 write (10,000 × $6.25/1M = $0.06) + 99 reads (990,000 × $0.50/1M = $0.50) = $0.56.

That's an ~89% reduction on the fixed portion of the prompt — before counting the latency win. The break-even is fast: with a short-lived (5-minute) cache, the 1.25× write premium is repaid after the second request; with a 1-hour cache (2× write cost), after the third. Anything beyond that is pure savings.

Rule of thumb: if a chunk of your prompt is identical across 3 or more requests within the cache window, cache it. If it changes every time, don't — you'll pay the write premium for nothing.

Claude vs OpenAI vs Gemini: Prompt Caching Compared

All three major providers support prompt caching in 2026 and discount cached reads heavily, but they differ in whether you control caching, whether writes cost extra, and whether storage is billed.

The practical takeaways:

• OpenAI is the easiest — caching is on by default with no write fee, so it wins for low-repetition workloads. You optimize it purely by ordering your prompt (static first, dynamic last).
• Anthropic gives you the most control via cache_control breakpoints and a small write premium, which pays off in high-repetition workloads like agents and long chats.
• Gemini offers automatic implicit caching plus explicit context caching, but explicit caches add a per-hour storage fee — great for one huge document reused for a while, less so for spiky traffic.

For a broader look at picking between these providers, see our Claude vs GPT vs Gemini comparison. Always confirm current numbers against the official docs — pricing shifts: Anthropic prompt caching, OpenAI prompt caching, and Google Gemini context caching.

How to Implement Prompt Caching (Claude Example)

With Anthropic's API, you add a cache_control marker to the last stable block you want cached — the model caches everything up to and including that point.

Here's a Python example that caches a large system prompt so it's only billed at full price once:

from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LARGE_SYSTEM_PROMPT,  # e.g. 10K tokens of stable instructions
            "cache_control": {"type": "ephemeral"},  # cache everything up to here
        }
    ],
    messages=[{"role": "user", "content": user_question}],  # varies each request → not cached
)

# Confirm the cache is working
u = response.usage
print(u.cache_creation_input_tokens, u.cache_read_input_tokens, u.input_tokens)

On the first call you'll see tokens under cache_creation_input_tokens; on every repeat within the cache window they move to cache_read_input_tokens at ~10% of the price. Prompt caching is generally available on current Claude models — no beta header required — and you can place up to four cache breakpoints per request.

The same principle applies everywhere, even with no code: freeze the prefix, vary the suffix. In agent frameworks like LangChain or LlamaIndex, keep your tool definitions and system prompt byte-stable so the framework's requests all share one cacheable prefix.

The Cache-Miss Checklist: Why Your Cache Does Nothing

The number-one reason prompt caching "doesn't work" is a silent invalidator — a small dynamic value near the top of the prompt that changes the prefix bytes on every request.

Because caching is a strict prefix match, one changed byte before your checkpoint invalidates the entire cache after it. Audit your prompt-building code for these culprits — every one of them is a saving quietly leaking away:

How to verify: log cache_read_input_tokens on repeated requests. If it's zero when it shouldn't be, diff the exact rendered bytes of two consecutive prompts — the difference is your invalidator. This one habit catches the vast majority of "caching isn't saving me anything" problems.

Two more pro tips: - Pre-warm the cache at startup with a throwaway request so your first real user doesn't eat the cache-miss latency. - For bursty traffic with long gaps, use a longer TTL (Anthropic's 1-hour tier) or a scheduled re-warm, so the cache doesn't expire between bursts.

Common Mistakes to Avoid

• Caching a prefix that changes every request. You pay the write premium and never get a read. If the first tokens vary, don't cache.
• Putting the checkpoint at the very end of the whole prompt. For a shared preamble with a varying question, mark the end of the shared part — not the end of the request — or every call writes a distinct entry that's never reused.
• Interpolating "current date" or user data into the system prompt. It feels harmless and silently kills caching for everything downstream.
• Assuming caching hurts quality. It doesn't. The output is identical — this is a common myth that keeps teams from turning it on.
• Ignoring the response usage fields. If you're not reading cache_read_input_tokens, you have no idea whether caching is actually working.

Frequently Asked Questions

What is prompt caching?

Prompt caching is an LLM feature that stores the model's computed state for a repeated prompt prefix so it doesn't reprocess those tokens on future requests. When a new request starts with the same prefix, the model loads the saved state instead of recomputing it — cutting input token cost by up to 90% and often lowering latency. The output is identical; only the prefill computation is skipped.

How much does prompt caching save?

Cached input tokens are billed at roughly 10% of the normal input price on current flagship models, so prompt caching can cut input costs by up to 90% on repeated calls. The larger and more frequently reused your fixed prefix — a long system prompt, tool definitions, or a document reused across many questions — the bigger the saving. Real-world reductions of 50–90% on total spend are common.

Does prompt caching change the model's output?

No. Prompt caching only skips the prefill computation for the repeated prefix — the model still generates a fresh response each time. The output is byte-for-byte the same as an uncached request. Caching affects only cost and latency, never answer quality.

Why is my cache read showing zero tokens?

A zero cache-read count almost always means a silent invalidator changed the prefix. The usual causes are a timestamp or UUID near the top of the prompt, non-deterministic JSON serialization, a system prompt that varies per user, or a prompt below the model's minimum cacheable length. Because caching is a prefix match, one byte change anywhere in the prefix invalidates everything after it.

Do OpenAI, Anthropic, and Gemini all support prompt caching?

Yes, but differently. OpenAI caches automatically with no code changes and no write fee. Anthropic requires explicit cache_control markers and charges a small write premium in exchange for precise control. Google Gemini offers automatic implicit caching plus explicit context caching that adds a per-hour storage fee. All three discount cached reads heavily on current models.

How long does a prompt cache last?

Cache lifetime is short and provider-specific. Anthropic's default cache lives about 5 minutes and refreshes each time it's read, with an optional 1-hour tier. OpenAI's automatic cache typically persists for several minutes of inactivity. For bursty traffic with long idle gaps, use a longer TTL or pre-warm the cache before traffic arrives.

When is prompt caching not worth it?

Skip caching when the first part of your prompt changes on every request — there's no reusable prefix, so you only pay the write premium with no reads. It also won't help if your prompt is below the model's minimum cacheable size, or if repeated requests arrive so far apart that the cache always expires between them.

Conclusion

Prompt caching is the rare optimization that costs almost nothing to adopt and pays back within a handful of requests. The mental model is simple: you're a prefix match away from a 90% discount — freeze the stable parts of your prompt, keep the volatile parts last, and confirm it's working by watching cache_read_input_tokens.

Start today with the highest-repetition part of your workload: a long system prompt, your agent's tool definitions, or a document you query many times. Turn caching on, audit for silent invalidators, and measure the drop. For most teams that single change is worth more than any model downgrade — and unlike a downgrade, it costs you nothing in quality.

Building or scaling an LLM application and watching the bill climb? SolutionGigs connects you with vetted AI engineers who architect caching, RAG, and agent pipelines for production. Post your project on solutiongigs.in today — it's free to post →

Mohammed Yaseen

Founder, SolutionGigs

Mohammed builds and optimizes production LLM applications — from prompt caching and RAG pipelines to agents and cost tuning. He founded SolutionGigs to connect teams with engineers who ship efficient, reliable AI systems. LinkedIn →