Glossary
Prompt caching
Reusing the already-processed start of a prompt across requests to cut latency and cost.
Prompt caching is a feature of many model APIs that reuses the work already done on an identical prompt prefix, such as a long system prompt or a large document, so repeated requests are faster and cheaper. The cached part must be the same, and must come first in the prompt.
It helps most when many requests share a large, stable block: instructions, tool definitions, a reference manual or a long conversation. Put stable content at the start and the changing question at the end, so the prefix stays identical.
Providers differ in how caching is switched on, how long entries last and how cached tokens are billed, so check the current documentation. Cached tokens are usually priced differently; see the LLM pricing page.
Start in 60 seconds
Put this to work on your project.
Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.