Context window: why AI "forgets" the start of a long conversation

A context window is the maximum amount of text (measured in tokens, roughly word fragments) an AI model can "hold" in view at once while generating a response. Anything beyond that window is physically unavailable to the model — not because it "forgot" in a human sense, but because that information simply isn't part of its current input.

Why it works this way

Unlike a person, an AI model doesn't retain long-term memory between individual messages on its own — with every new reply, it reprocesses the entire conversation text available to it, from the start of the chat to the latest message (up to the window's limit). Once a conversation's length exceeds the window size, the oldest messages stop making it into the model's input — and it responds as though they never happened.

Why different models have different limits

Context window size is a specific technical spec of a model that can vary widely: from a few thousand tokens on older or more compact models to hundreds of thousands on modern flagship systems. A larger window requires significantly more compute per request — so it's not simply a matter of "improve it and make it unlimited," but a direct tradeoff between capability and cost.

What this means in practice

If you're having a long working conversation with an AI and notice it starts confusing or ignoring details mentioned earlier, the likely cause is the context window filling up, not the model "getting worse." A practical fix is to periodically summarize key agreements in a new message so they're guaranteed to stay within the active part of the window.

This material is for educational purposes only.