What Is Context Caching? Reusing Work to Speed Up Repeated Prompts
When a large chunk of a prompt stays the same across many requests, context caching avoids reprocessing it every single time — cutting both latency and cost.
Not reprocessing what hasn't changed
Context caching stores the internal computation a model already did for a piece of context — like a long system prompt or reference document — so that when the same content appears again in a later request, the model can reuse that stored work instead of processing it again from scratch.
Why this matters for cost and speed
Processing a long prompt takes real compute every single time, even if most of it is identical to the last request. When a large, stable chunk (instructions, a reference document, a long conversation history) repeats across calls, caching that portion can meaningfully cut both response latency and per-request cost.
What makes content cacheable
Caching works best on content that's genuinely stable and reused across requests — a fixed system prompt or a document referenced repeatedly — rather than content that changes on every call, since a cache only helps when there's actually something identical to reuse.
Frequently Asked Questions
What is context caching?
A technique that stores a model's internal computation for a piece of context so it can be reused across later requests that include the same content, instead of reprocessing it from scratch each time.
What kind of content benefits most from context caching?
Large, stable content that repeats across many requests — like a long system prompt or a frequently referenced document — since caching only helps when there's genuinely identical content to reuse.