StackLift
All terms

Glossary

Context window

The maximum amount of text, in tokens, a model can take in and produce in a single request.

A context window is the total number of tokens a language model can handle in one request, counting your instructions, any documents you include, the conversation so far and the answer it writes. It is the model's working memory for that request, not permanent memory.

Context windows have grown a great deal, but a bigger window does not mean every part of it is used equally well. Models can miss details buried in the middle of very long inputs, and long inputs cost more and respond more slowly.

When your content does not fit, or is mostly irrelevant, teams retrieve only the useful passages with RAG or summarize earlier turns. Context sizes per model are listed on the LLM pricing page.

Start in 60 seconds

Put this to work on your project.

Get an AI-assisted, architect-reviewed scope, cost range, and timeline. Free, no login.

The complete guide

Context window: the complete guide

Updated November 18, 2026 ยท Published by , Founder, StackLift AI. Written with AI assistance; how we make content.

Short answer

A context window is the total amount of text, measured in tokens, that a language model can handle in a single request. It includes your instructions, any documents, the conversation so far and the model's answer. It is working memory for that request only, not permanent memory.

What is a context window?

Think of the context window as the desk the model works on. Everything it can see for this request has to fit on the desk: your system prompt, tool descriptions, retrieved passages, earlier chat turns, your question and the answer it writes.

It is measured in tokens. When a request needs more than the window allows, it fails or older content gets dropped. The next request starts with an empty desk unless your application sends the earlier content again.

What counts toward the context window?

Parts of a typical request
PartExampleCounts toward the window?
InstructionsSystem prompt, rules, toneYes
Tool definitionsDescriptions of functions the model may callYes
Supplied contentRetrieved passages, pasted documentsYes
Conversation historyEarlier user and assistant turnsYes
The answerText the model writes backYes, shared with the input

Many providers also set a separate cap on output length. Check each model's documentation for both numbers; context sizes per model are listed on the LLM pricing page.

Is a bigger context window always better?

No. Larger windows let you include more, but they come with trade-offs. Cost rises with every token sent, responses get slower, and models can miss details placed in the middle of very long inputs. Filling the window with loosely relevant text can make answers worse, not better.

Treat the window as a budget. The goal is to put the most useful content in front of the model, not the most content.

How do you work within the limits?

Common techniques
TechniqueWhat it doesBest for
RAGRetrieves only the passages relevant to the questionLarge or changing document sets
Summarizing historyReplaces old turns with a short summaryLong conversations and agent runs
Chunking and map-reduceProcesses a long document in parts, then combines resultsOne-off analysis of very long files
Prompt cachingReuses a repeated prefix to cut cost and delayStable instructions and reference text
Trimming tool outputPasses back only the fields the model needsAgents calling APIs

What is context engineering?

Context engineering is the practice of deciding what goes into the window at each step: which instructions, which retrieved documents, which tool results and how much history. It extends prompt engineering from writing a good prompt to managing everything the model sees.

It matters most for AI agents, which run many steps and can fill the window with tool output. Good agents keep context small, structured and relevant, and log what they send so failures can be traced.

Is the context window the same as memory?

No. The window holds what is sent in the current request. Memory features in assistants are built by the application: it stores facts or summaries somewhere, such as a database, and puts the relevant ones back into later requests.

That distinction matters for privacy and design. Decide what is stored, for how long, who can see it and how it is deleted. StackLift plans this with the LLM integration scope; the free estimate is a good start.

How should context size affect your model choice?

Context size is one factor among several. Check that the window fits your longest realistic request, including instructions, retrieved passages and the answer, with room to spare. Then test whether the model uses long inputs well, by placing the key fact at different positions and seeing if it is found.

If most requests are short, a larger window gives you little. If you work with long contracts or codebases, it can simplify design, but cost and speed still matter. Compare options on the LLM pricing page, and keep the model behind a thin layer so you can switch.

How does a context window work?

A model does not remember earlier conversations on its own. Each request carries everything the model should know, and the context window is the space that holds it. When the space is full, something has to be dropped or shortened.

  1. Your application collects the inputs: system instructions, the user's message, earlier turns of the conversation and any retrieved documents or tool results.
  2. The text is split into tokens, the small pieces a model reads.
  3. The model reads all of those tokens at once and writes a reply, which also uses tokens from the same window on most models.
  4. The total of input and output must fit within the limit. Many providers also set a separate cap on the reply length.
  5. On the next turn, your application sends the history again, so the window fills as the conversation grows.

This is why a chatbot that seems to remember you is actually being handed a transcript every time. See agent memory for how teams add durable memory on top.

What takes up space in a context window?

Parts of a typical request
PartWhat it holdsGrows when
System promptRules, role, tone and format instructionsYou add more policies and examples
Conversation historyEarlier user and assistant turnsThe chat gets longer
Retrieved contentPassages pulled in by RAGYou retrieve more or longer chunks
Tool definitions and resultsDescriptions of available tools and what they returnedYou add tools or tools return large payloads
User inputThe current question or uploaded fileUsers paste long documents
OutputThe reply, plus hidden reasoning on some modelsYou ask for long or detailed answers

Tool results are the sneaky one. A single database query or web page can return far more text than the question needs, and an agent loop adds results turn after turn.

Where do context window limits show up in real products?

  • Document question answering: a contract review tool must decide whether to send the whole contract or only the clauses that match the question.
  • Support assistants: a long customer thread plus order history and policy text can crowd the window, so the app summarizes the thread.
  • Coding assistants: the model cannot see an entire repository, so the tool selects the files and functions that matter.
  • Research agents: each web page or file read adds to the window, so the agent keeps notes and drops raw pages.
  • Meeting and call analysis: a long transcript may need to be processed in parts and then combined.

In each case the design question is the same: what is the smallest set of information that lets the model do this step well?

What mistakes do teams make with context?

Mistakes and fixes
MistakeResultFix
Stuffing everything inHigh cost, slow replies, missed detailsRetrieve and trim
Ignoring the output budgetReplies cut off midwayReserve room for the answer
No plan for long chatsErrors once the limit is hitSummarize or window the history
Assuming the model remembersUsers repeat themselves across sessionsStore memory outside the model
Testing only short inputsFailures appear in productionTest with realistic, long cases
Placing key rules far from the questionInstructions are weakly followedKeep critical instructions clear and repeat when needed

What does the context window mean for cost and timeline?

Context drives running cost more than most buyers expect. Each request is priced by the tokens it carries, so a design that resends large inputs on every turn costs more to run than one that retrieves small pieces. It also affects response time and the engineering needed for retrieval, summarizing and caching.

When you plan an AI feature, ask how much context each request needs and how it will be kept small. The LLM cost optimization playbook and the AI LLM integration cost page cover the drivers, and the free estimate shows how an AI feature fits into a plan.

How does StackLift plan for context limits?

When we scope an AI feature, we ask how much context each request truly needs, where it comes from and what it costs to run. Architects review that choice before it becomes a commitment, and engineers own the retrieval and trimming design. AI agents speed the work, and they work on feature branches, never on the default branch.

Evaluation cases with long, realistic inputs go into the plan from the first sprint, so limits show up in testing and not in front of your users. You can see how scope, cost and team fit together by running the free estimate, or read how an AI scope sprint turns a rough idea into a plan.

How do you test a feature against context limits?

Context problems rarely appear in a demo, because demos use short, tidy inputs. They appear when a real customer pastes a long email chain or an agent runs for twenty steps. Testing for them is straightforward once you decide to do it.

  1. Collect the longest and messiest real inputs you expect, and add them to your evaluation set.
  2. Measure how many tokens each part of the request uses: instructions, history, retrieved passages, tool output.
  3. Set a budget for each part, and a limit on the total with room left for the reply.
  4. Run long conversations and long agent loops to see what happens at the limit.
  5. Check that key instructions are still followed late in a long request.
  6. Compare answer quality with a trimmed context and a full one, and keep the smaller version if it performs as well.

Watch the token counts in production too. A slow rise in average request size is an early warning of rising cost and falling quality.

Questions people ask

What is the context window of ChatGPT or Claude?

It depends on the model and the plan, and it changes as new models are released. Check the provider's documentation or the model list on our LLM pricing page.

What is the difference between context window and tokens?

Tokens are the unit of text. The context window is the maximum number of tokens a model can handle in one request.

What happens when the context window is full?

The request errors out, or the application drops or summarizes older content. Well-built products manage this deliberately rather than letting it happen at random.

Does a long context window replace RAG?

Not usually. Long windows help with one-off analysis, but RAG keeps cost down, supports citations and scales to content that never fits in one request.

Why does the model forget things I said earlier?

Either the earlier text was dropped to make room, or the model gave it less attention among a lot of other content. Restating key facts or summarizing helps.

Is the context window the same as memory?

No. The context window holds what the model can see in one request. Lasting memory is something your application stores and feeds back in, such as a summary or a profile.

What happens when I exceed the context window?

Depending on the provider, the request is rejected with an error, or older content must be removed before sending. Good applications trim or summarize before they reach the limit.

Does a larger context window replace RAG?

Not usually. RAG sends only relevant passages, which keeps cost low and answers focused. A large window helps when you truly need to reason over a whole document at once.

Sources

Sources and further reading

The standards, rules, research and official documentation this page draws on.