Context Window

What is a context window?

A context window is the information available to a language model for one inference. It can include system instructions, the current user request, conversation history, retrieved documents, tool results, and other text the application sends to the model.

The model does not automatically remember everything that happened before. An agent system has to decide what information belongs in the current context and what should live somewhere else, such as persistent memory or an external knowledge store.

How it works

Text is converted into tokens before inference. The model can only consider a bounded amount of tokenized input and generated output within the limits of the model and API configuration.

For an AI agent, that makes context management an architectural problem. The system may need to summarize old history, retrieve only relevant knowledge, trim verbose tool outputs, or separate persistent state from temporary working context.

Simple example

Imagine a research agent that has already read twenty long documents. Sending every document again on every step would waste context space and increase cost. A better design can keep the documents in external storage, retrieve the sections relevant to the current question, and place only those sections into the active context.

Common confusion

A context window is not the same thing as agent memory. Context is what the model can see during the current inference. Memory is a broader system capability for storing information and retrieving it again when useful.

A larger context window can reduce some context-management pressure, but it does not remove the need to decide what information is relevant, trustworthy, current, and worth including.

Why it matters

Poor context design can make an agent slower, more expensive, and less reliable. Good context design gives the model the instructions, state, evidence, and tool observations it needs without flooding it with irrelevant material.