Context Window
Last Updated: September 10, 2026 | By Mihail Sebastian | AI Dictionary
The maximum text a large language model can consider at once, measured in tokens: its working memory for conversations, documents, and tool results.
What is a Context Window?
A context window is the maximum amount of text a large language model can consider at once, measured in tokens. It works as the model’s working memory: everything the model knows about your session must fit inside it.
The model has no memory beyond the window. What looks like remembering is rereading: on every turn, the application sends the conversation so far back into the window.
How the Context Window Works
Input text is split into tokens through tokenization; a token is roughly a short word or a piece of a longer one. The window is a fixed token budget shared by everything the model needs: the developer’s system instructions, the conversation history, any pasted documents, and, for agents, the results of every tool call.
When the budget runs out, something has to go. Applications truncate or summarize the oldest content, so the model does not fail; it quietly loses whatever was dropped. And a large window is not a guarantee the model uses all of it equally well: research on long inputs has found that details buried in the middle are recalled less reliably than material near the start or end.
Because context is finite, applications retrieve rather than load. Retrieval-augmented generation is the standard pattern: instead of pasting an entire knowledge base into the window, the system fetches only the passages relevant to the current question.
Example of a Context Window
A compliance officer spends an afternoon reviewing a vendor contract with an AI assistant, pasting in clauses and asking questions. Early in the session she states the key constraint: “we cannot accept liability caps below two million.”
Hours later the conversation has grown past the token budget, and the application silently drops the oldest turns to make room. When she asks for a final recommendation, the constraint is no longer in the window, and the assistant endorses a clause with a lower cap.
The model did not disagree with her. It never saw the instruction, and nothing in its answer says so. Teams that run assistants on long documents watch for exactly this failure: what the model was actually shown, not what the user believes it was shown.
FAQ
What happens when the context window is full?
The application decides, not the model. Most truncate or summarize the oldest content and keep going, so the visible symptom is not an error but an assistant that has forgotten earlier parts of the session.
Does a bigger context window remove the need for RAG?
Not in practice. Even very large windows are finite, filling them costs money and time on every request, and retrieval keeps the model focused on relevant passages instead of asking it to find them in a haystack.
Related AI terms: Tokenization · RAG · Large Language Model · Hallucination · AI Agent
Did you like the Context Window gist?
Learn about 250+ need-to-know artificial intelligence terms in the AI Dictionary.
