StackLift
All terms

Glossary

Tokens

The small chunks of text a language model reads and writes, and the unit most AI APIs bill by.

Tokens are the pieces of text a language model actually processes: whole words, parts of words, punctuation or single characters. Model providers count input and output in tokens, so tokens determine both how much fits in a request and what a request costs.

A tokenizer splits text before the model sees it. Common words are often one token, while rare words, code, numbers and many non-English languages use more tokens for the same meaning. Each vendor uses its own tokenizer, so counts differ between model families.

Because input and output both count, long prompts and long answers raise cost and slow responses. Live per-token prices are on the LLM pricing page.

Start in 60 seconds

Put this to work on your project.

Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.

The complete guide

Tokens: the complete guide

Updated November 18, 2026 ยท Published by , Founder, StackLift AI. Written with AI assistance; how we make content.

Short answer

Tokens are the small pieces of text a language model reads and writes: whole words, parts of words, punctuation or single characters. AI providers count input and output in tokens, so tokens set both how much text fits in one request and how much that request costs.

What is a token in AI?

A large language model does not read letters or words. A tokenizer first splits your text into pieces from a fixed vocabulary, and each piece becomes a number the model works with. A token can be a whole common word, a fragment of a longer word, a space with punctuation, or a single character.

Providers often give a rule of thumb that a token is roughly a short word or three-quarters of one in English. Treat that as a rough guide only, since each vendor uses its own tokenizer and counts differ.

Why do tokens matter?

Where tokens show up in a project
AreaWhat tokens control
CostAPIs bill per token, usually with separate rates for input and output
LimitsYour prompt plus the answer must fit in the context window
SpeedLonger outputs take longer, because the model writes one token at a time
Rate limitsProviders cap tokens per minute as well as requests per minute

Live per-token prices for each model are on the LLM pricing page. Prices change often, so build cost checks into your product rather than hard-coding assumptions.

What is the difference between input and output tokens?

Input tokens are everything you send: instructions, retrieved documents, tool definitions, the conversation so far and the new question. Output tokens are what the model writes back. Providers usually price output tokens higher than input tokens, because generating text takes more compute than reading it.

Some models also produce reasoning tokens while they think before answering. These are typically billed as output even when you do not see them in full, so reasoning models can cost more per answer than their visible text suggests.

Why does the same text use different token counts?

  • Different vendors use different tokenizers, so one model's count is not another's.
  • Code, numbers, URLs and unusual names split into more tokens than plain prose.
  • Many non-English languages need more tokens for the same meaning than English does.
  • Whitespace and formatting, such as long JSON, add tokens.

Use the provider's own token-counting tool or API response, which reports usage, instead of estimating by word count when cost or limits matter.

How do you reduce token use?

  1. Cut instructions to what the model needs; remove old rules that no longer change the answer.
  2. Retrieve only relevant passages with RAG instead of pasting whole documents.
  3. Summarize older conversation turns rather than resending them in full.
  4. Ask for concise or structured output, and set a maximum output length.
  5. Reuse repeated prompt prefixes with prompt caching.
  6. Route simple tasks to a smaller, cheaper model.

Measure first. Log input and output tokens per request and per feature, so you cut the costly step rather than guessing.

How should you plan for token costs in a project?

Running cost depends on how many requests you expect, how long each prompt and answer is, and which model handles each step. Agents multiply this, since one task may involve many model calls and tool results fed back in.

StackLift scopes this as part of an estimate: usage assumptions, model choice, monitoring and cost limits are planned alongside the build. The free estimate and the LLM integration page show how the pieces fit.

Why do agents use so many tokens?

An agent works in a loop: it plans, calls a tool, reads the result and plans again. Each loop resends the instructions, the tool definitions and the growing history, so token use grows with the number of steps. A task that looks like one request can involve many model calls.

Keep tool output short, summarize history and set step limits. Logging tokens per task shows which step costs the most, and routing easy steps to smaller models helps. See AI agents for how they are built, and the AI agent cost guide for what drives cost.

How does a model turn text into tokens?

Before a model reads anything, a tokenizer splits the text into pieces from a fixed vocabulary and maps each piece to a number. Most modern tokenizers use a method called byte-pair encoding, which builds the vocabulary from frequent sequences of characters. Common words become one token, rarer words are split into several, and unusual strings fall back to small fragments.

  1. Your text arrives as characters.
  2. The tokenizer splits it into tokens using its learned vocabulary.
  3. Each token becomes an ID number.
  4. The model processes the IDs and predicts the next ID.
  5. The predicted IDs are converted back into text for you.

Each model family has its own tokenizer, so the same sentence can count differently between providers. That is why a count from one vendor's tool does not transfer exactly to another's.

What kinds of content use tokens?

What consumes tokens in a request
ContentHow it countsPlanning note
Plain textWords and parts of words become tokensTypical for chat and summaries
Code and data (JSON, XML)Punctuation, brackets and indentation add tokensStructured text often costs more per visible character
Non-English textSome languages split into more tokens per wordTest your own languages before budgeting
ImagesAn image is converted into a number of tokens that depends on its sizeResize before sending where detail is not needed
Audio and videoConverted into tokens or priced by length depending on the modelCheck the provider's rules
Tool definitions and system promptsSent with every request and counted each timeTrim them and consider prompt caching
Reasoning (thinking) outputHidden steps may be billed as output tokensSet limits on reasoning effort

The takeaway is that the visible message is only part of what you pay for. Instructions, history, retrieved documents and hidden reasoning all count.

Which everyday features use the most tokens?

Feature design drives token use more than any setting. Here are common patterns, from light to heavy.

  • Short classification of an incoming message: a small prompt and a one-word answer.
  • Customer chatbot: a standing prompt plus a growing conversation history on every turn.
  • Document summarization: the entire document goes in, a short summary comes out.
  • Retrieval (see RAG): the question plus several retrieved passages on every call.
  • Code assistant: large parts of a codebase placed in context.
  • Agent loop: each step repeats the history and the tool results, so cost grows with the number of steps.

Knowing which pattern you are building tells you where to look first when the bill is higher than planned.

How do you track whether token use is under control?

  • Tokens per request, split into input, output and cached input.
  • Cost per successful task, such as per resolved ticket or per processed document.
  • Share of requests that hit the context limit or are truncated.
  • Cache hit rate where prompt caching is used.
  • Spend by feature, model and customer, so you can see who and what drives it.
  • Retries and failed calls, which spend tokens and deliver nothing.

Log these from day one. A feature that looks cheap in a demo can behave very differently with real usage, and the logs are how you find out early. The AI cost optimization entry and the LLM cost optimization playbook go further.

What tools count and manage tokens?

Providers publish tokenizers or counting endpoints: OpenAI's tiktoken library, Anthropic's token counting API and Google's count tokens method for Gemini. Gateways and observability tools such as LiteLLM, Langfuse and OpenTelemetry-based tracing record usage per call. Use them in development to estimate cost before launch and in production to watch it.

What do tokens mean for project cost and timeline?

Token cost is a running cost, separate from the build. It depends on how many requests you expect, how long each prompt and answer is, which model you choose and whether you can cache or batch work. Cheaper models handle routine steps, while stronger ones are saved for hard cases, which is the idea behind model routing.

Current rates for each model are on the LLM pricing page. The cost to build an AI app guide explains how running costs sit beside build costs, and the free estimate gives a project-level view.

What mistakes do teams make with tokens?

  • Sending the full conversation history on every turn instead of summarizing older turns.
  • Pasting whole documents into the prompt when a few relevant passages would do.
  • Ignoring output length, so answers run long and cost more than needed.
  • Estimating cost from the demo, not from realistic traffic and realistic inputs.
  • Leaving agent loops without a step limit, so one stuck task spends a large amount.
  • Counting words and assuming the provider's count will match.

Most of these are design choices made early and never revisited. A short review of prompt and history size before launch usually pays for itself.

How does StackLift plan for token costs?

On AI projects, StackLift treats running cost as part of the design, not an afterthought. Scope is reviewed by a senior architect, and the plan includes choices such as which model handles which step, what is cached, and where limits apply. Model prices are tracked at LLM pricing, so estimates reflect current rates.

Observability is part of the plan from sprint one, so usage per feature is visible as the product grows. If you are weighing an AI feature, the free estimate scopes it feature by feature with no signup.

Questions people ask

How many tokens are in a word?

It varies. Common English words are often a single token, while longer or rarer words, code and other languages use several. Use your provider's token counter for real numbers.

Are tokens the same as words?

No. A token can be a word, part of a word or punctuation. Word count and token count are related but never identical.

Do I pay for input and output tokens?

Yes. Most providers bill both, usually at different rates. Cached or batched requests may be billed differently, so check each provider's current terms.

What happens when I exceed the token limit?

The request is rejected, or the output is cut off at the limit. Shorten the input, retrieve less, or choose a model with a larger context window.

Do all models count tokens the same way?

No. Each model family has its own tokenizer, so the same text can produce different counts across vendors.

Are tokens the same as words?

No. A token is a piece of text, often part of a word. Short common words are usually one token, and long or unusual words are split. Counts differ by language and by model.

Do I pay for tokens I send and tokens I receive?

Yes, most providers bill both, usually at different rates for input and output. Some also bill cached input differently and may count hidden reasoning as output.

What happens when I exceed the token limit?

The request may fail, or the output may stop early, depending on the provider and settings. Design for it: trim history, summarize older turns, and retrieve only what is relevant.

Can I reduce tokens without hurting quality?

Often. Shorter instructions, selective retrieval, caching repeated prefixes and routing easy tasks to smaller models all cut use. Test each change on your own examples to make sure quality holds.

Sources

Sources and further reading

The standards, rules, research and official documentation this page draws on.