StackLift
All playbooks

Playbook

LLM Cost Optimization Playbook

LLM bills follow tokens, model choice, and call volume. This playbook gives you the order to work in: measure first, then trim, route, cache, and batch.

Best reader

Engineering leads and product owners running LLM features in production

Outcome

A measured, quality-checked plan that lowers cost per task and keeps it from drifting back up.

Use this sequence

1

Measure token use and cost per feature, user, and task.

2

Set a quality baseline on a test set before changing anything.

3

Trim prompts, context, and output length.

4

Route simple tasks to smaller models and hard ones to stronger models.

5

Cache repeated prompt prefixes and repeated answers where allowed.

6

Move work that can wait to batch processing.

7

Set budgets and alerts, and review cost after each release.

Measure before you cut

You cannot lower a cost you cannot see. Log input tokens, output tokens, model, and feature on every call, then find the few features that make most of the spend.

Cost per task

Tokens per call

Retries and failures

Tool-call loops

The main levers

Spend falls through fewer tokens, cheaper models for easy work, caching, and batching. Each lever risks quality, so test each against your baseline.

Shorter prompts and context

Model routing

Prompt caching

Batch processing

Guard quality and keep watching

A cheaper model that gives worse answers costs more in rework. Pair every change with an evaluation run, and set per-key budgets so a runaway loop cannot surprise you.

Evaluation gate

Budget alerts

Prices tracked at /llm-pricing

Start in 60 seconds

Turn this checklist into a scoped plan.

Answer a few questions and get an AI-assisted, architect-reviewed scope, cost range, and timeline for your software project.

Frequently asked questions

What is the first thing to check when an LLM bill is too high?

Find which feature and which model make most of the spend, then look for oversized context, repeated calls, and agent loops. Those are usually behind the biggest jumps.

Does using a smaller model hurt quality?

Sometimes, and sometimes not for simple tasks. Test the smaller model on your own examples and route only the tasks where it passes.

The complete guide

LLM Cost Optimization Playbook: the complete guide

Updated November 18, 2026 ยท Published by , Founder, StackLift AI. Written with AI assistance; how we make content.

Short answer

To lower LLM costs without hurting quality, measure tokens and cost per feature first, set a quality baseline, then work the main levers in order: trim prompts and context, route easy tasks to smaller models, cache repeated prompt prefixes, batch work that can wait, and cap usage with budgets. Test each change against your own examples.

What drives the cost of an LLM feature?

Most providers charge by tokens, which are small pieces of text. You pay for input tokens (your prompt, context, and history) and output tokens (the reply), usually at different rates per model. Your bill is the number of calls, times the tokens in each, times the price of the model you used.

That formula points at the levers: fewer calls, fewer tokens per call, and cheaper models where they are good enough. Agents add a twist, because one user task may trigger many model calls and tool results that all get fed back as input. Live prices change often, so check LLM pricing when you model a budget rather than relying on numbers in an article.

What should you measure before cutting costs?

Start by finding where the money goes. Log the model, feature, user or account, input tokens, output tokens, and outcome on every call. A few features usually produce most of the spend, and they are where effort pays.

Cost measures to log
MeasureWhy it matters
Cost per task or conversationThe unit that links spend to value
Input and output tokens per callShows oversized prompts and long replies
Calls per taskReveals agent loops and retries
Model used per featureShows where a smaller model might do
Cache hit rateTells you whether caching is working
Failure and retry rateFailed calls still cost money
Quality score on a test setKeeps savings honest

Pair the cost data with a quality baseline. Without one, you cannot tell a saving from a regression. The AI agent evaluation checklist describes how to build the test set and rerun it on each change.

In what order should you optimize?

  1. Instrument every call and find the top cost drivers. Output: a ranked list of features and models by spend.
  2. Set a quality baseline on real examples. Output: a score you must not fall below.
  3. Remove waste: duplicate calls, oversized context, runaway loops. Output: the same results with fewer tokens.
  4. Tighten prompts and cap output length. Output: shorter inputs and outputs that still pass.
  5. Route tasks to the smallest model that passes. Output: a routing table per task type.
  6. Cache stable prompt prefixes and repeated answers. Output: lower cost on repeat work.
  7. Move non-urgent work to batch processing. Output: background jobs at a lower rate where offered.
  8. Set budgets, alerts, and per-account limits. Output: guardrails against surprises.
  9. Review cost after each release. Output: a routine, not a one-off project.

Write the result of each step down: what you changed, the cost before and after on your own traffic, and the quality score on the test set. A short log like this tells you which levers worked for your workload, and it protects you from repeating a change that quietly hurt quality. It also gives finance and product a shared record of why the bill moved.

The order matters. Removing waste is nearly free and risks no quality. Model changes carry the most quality risk, so do them after you have a baseline and a test suite.

What are the main levers, and what do they risk?

LLM cost levers compared
LeverHow it savesQuality riskBest for
Trim prompts and contextFewer input tokens per callCutting something the model neededLong system prompts and chat history
Limit output lengthFewer output tokensTruncated or terse answersFeatures with a fixed answer shape
Model routingCheaper models handle easy tasksA small model fails on hard casesMixed traffic with simple and hard requests
Prompt cachingRepeated prompt prefixes cost less to processLittle, if prompts are unchangedLarge stable prefixes reused across calls
Batch processingProviders often price asynchronous batches lowerResults arrive laterReports, enrichment, nightly jobs
Response cachingSkips the call for repeat questionsStale or wrong-user answersCommon questions with stable answers
Retrieval over long contextSends only the relevant textRetrieval misses the key passageLarge knowledge bases; see RAG
Fine-tuning or a private modelCan reduce prompt size or per-call price at scaleSetup and upkeep effortHigh, steady volume of one task

Provider features and discounts differ and change, so confirm the current terms in each provider's documentation before you plan around them.

How do you trim prompts and context safely?

Every token you send is paid for on every call. Long system prompts, whole documents pasted for one fact, and full chat history are common sources of waste. Trim in small steps and test after each step.

  • Remove repeated instructions and examples the model no longer needs.
  • Send only the relevant passages from documents, using search or retrieval.
  • Summarize older chat turns instead of resending them in full.
  • Return only the fields a tool needs, not the whole record.
  • Ask for structured, short output, and set a maximum length.
  • Keep stable text at the start of the prompt and changing text at the end, which helps caching.

Re-run your test set after each change. If quality falls, put back the last thing you removed.

How does model routing work?

Routing sends each request to the cheapest model that can handle it. A simple classification or extraction may be fine on a small model, while multi-step reasoning or sensitive writing may need a stronger one. Test each task type on your own examples before you decide.

Routing approaches
ApproachHow it worksTrade-off
By task typeFixed rules map each feature to a modelSimple and predictable, but not adaptive
By classifierA small model labels difficulty, then routesAdds a call, can misjudge hard cases
CascadeTry a small model, escalate if confidence is lowMore latency on hard cases
By user tierDifferent models for different plansNeeds clear product rules

Add a fallback so a failure on one model does not stop the feature. Keep the routing table in configuration so you can change models as prices and capabilities move. Our LLM integration work includes this kind of design.

When do caching and batching help?

Prompt caching helps when many calls share a long, unchanged beginning, such as a big system prompt or a reference document. Providers that support it charge less to process the cached part. Batching helps when a result can wait: you submit many requests together and collect results later, often at a lower rate.

  • Cache: stable instructions, tool definitions, and reference text that repeat across calls.
  • Do not cache: personal data that must not cross users, or answers that go stale quickly.
  • Batch: nightly summaries, bulk classification, data enrichment, and evaluation runs.
  • Do not batch: anything a user is waiting on in real time.
  • Check each provider's rules on cache lifetime, minimum size, and batch turnaround before you rely on them.

Caching a response is different from caching a prompt prefix. A response cache skips the call completely, so key it carefully by input and by who is asking, or one user may see another's answer.

How do you keep costs from creeping back?

Savings fade if nobody watches. Set budgets and alerts per feature and per account, with a hard cap for runaway loops. Review spend whenever you change a prompt, model, or tool, because those changes move the bill.

Cost guardrails
GuardrailWhat it does
Per-key or per-account budgetStops one user or script from running up spend
Maximum calls per taskEnds agent loops
Maximum tokens per callLimits oversized prompts and replies
Alert on daily spend changeFlags a regression early
Cost check in release reviewCatches a costly change before it ships
Monthly review of top featuresKeeps the ranked list current

When you plan a new AI feature, model its running cost along with the build. The cost guides show what drives build cost by project type, and the free estimate scopes a project feature by feature. To talk through an existing bill, contact us.

Questions people ask

How do you reduce LLM API costs?

Measure where the spend goes, then remove waste, trim prompts, route simple tasks to smaller models, cache repeated prompt prefixes, and batch work that can wait. Test every change against a quality baseline.

What is prompt caching?

It lets a provider reuse the processing of a repeated prompt beginning, so the repeated part costs less on later calls. It works best with long, stable prefixes. Check each provider's rules and lifetime.

Is a smaller model always cheaper overall?

Not always. If a smaller model needs more attempts or produces errors that need rework, the total can rise. Test on your own examples and count retries and fixes, not just the price per token.

How do you choose which model to use?

Test candidates on your own cases and compare quality, speed, and cost. Compare families and sizes, and use the live figures on the LLM pricing page. Pick the cheapest model that meets your quality bar for each task.

Why are agents expensive to run?

One task can trigger many model calls, and tool results get added to the input each time. Loops and retries multiply this. Limit steps per task and watch calls per task as a cost measure.

When does a private or fine-tuned model make sense?

When volume is high and steady for one narrow task, or when data rules require it. It adds setup and upkeep work, so test hosted options with good prompting and caching first.

How do you forecast LLM costs before building?

Estimate calls per task, tokens per call, and tasks per user, then price them with current model rates. Run a pilot to replace guesses with measured numbers, and add a margin for retries.

Sources

Sources and further reading

The standards, rules, research and official documentation this page draws on.