Glossary
Tokens
The small chunks of text a language model reads and writes, and the unit most AI APIs bill by.
Tokens are the pieces of text a language model actually processes: whole words, parts of words, punctuation or single characters. Model providers count input and output in tokens, so tokens determine both how much fits in a request and what a request costs.
A tokenizer splits text before the model sees it. Common words are often one token, while rare words, code, numbers and many non-English languages use more tokens for the same meaning. Each vendor uses its own tokenizer, so counts differ between model families.
Because input and output both count, long prompts and long answers raise cost and slow responses. Live per-token prices are on the LLM pricing page.
Start in 60 seconds
Put this to work on your project.
Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.