Context Window

The maximum number of tokens a model can consider at once — everything it is allowed to "see" for a single response.

Natural Language Beginner 1 min read

Also known as: Context Length

Definition

The context window is the hard limit on how much a model can attend to in one go, counted in tokens and shared between the prompt and the response. Exceed it and something has to be dropped.

Windows have grown from a couple of thousand tokens to hundreds of thousands, but bigger is not straightforwardly better. Attention cost grows quadratically with length, and models reliably attend better to the beginning and end of a long context than the middle — the "lost in the middle" effect. Filling a large window indiscriminately often performs worse than retrieving the right few thousand tokens.

How it relates to other terms

Prerequisite Token The unit a language model actually reads and writes — usually a word fragment rather than a word or a characte... See also Attention A mechanism that lets a model weigh how much each part of the input should influence each output, learned rath... See also Retrieval-Augmented Generation Fetching relevant documents at query time and putting them in the prompt, so the model answers from your data...
Referenced by
Large Language Model See also Token See also

Related terms

Large Language Model

LLM

A neural network trained on very large amounts of text to predict what comes next, which turns out to be enough to answer questions, write code and ho...

Beginner Natural Language

Token

The unit a language model actually reads and writes — usually a word fragment rather than a word or a character.

Beginner Natural Language

Chain-of-Thought

CoT

Prompting a model to work through its reasoning step by step before answering, which measurably improves multi-step problems.

Intermediate Natural Language

Was this definition helpful?

Definitions are reviewed by our editorial team. Spotted a problem? Tell us.

Keep reading

Browse the glossary Suggest a correction