← Back to concept explainers

Concept Fable

Context window

Conclusion first:Context Window is the total amount of input and output a model can process in a single request, usually measured in tokens. It determines the space that prompts, retrieved materials, historical messages, tool results, and generated answers can occupy together. Long-context models reduce truncation risk, but they do not automatically solve problems of relevance, ordering, contamination, cost, and attention dilution. Enterprise Agents need to make choices, ranking, compression, and budget management around the context window.

Context Window · LLM

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The judge who can see only one desk

In the old court there was a judge who read cases extremely fast, but he had one limitation: each time he could see only the papers on his desk. No matter how important the files outside the desk were, if they were not placed on it, he treated them as nonexistent.

In earlier years, cases were thin, and it was enough for the clerk to put the complaint, testimony and rules on the table. Later, cases involved contracts, old letters, historical complaints, tool receipts and precedents, and the desk began to get crowded.

The clerk first wrote smaller and crammed in more paper. The judge did see more material, but also more easily missed key sentences; new evidence and expired rules were squeezed together, and the order became chaotic.

They switched to an even larger desk. Everyone was happy for a while, and soon it was piled full again. A larger desktop let lazy organizing live longer and also let irrelevant materials enter judgment openly.

In one trial, the clause determining compensation was squeezed to the very end, and the judge did not see it. In another, an expired policy was placed in the most prominent spot and ended up influencing the verdict.

The new clerk changed the question: what must be known in this case, what is merely background, what should be summarized, what must be kept verbatim, and what should not be brought to the table at all. The desk was still limited, but the results became stable. The size of the boundary matters, but what you put on it matters more.

The clerk also learned to rank the materials. Rules that must be followed go in a fixed position, the current question sits right next to it, evidence is ordered by recency and relevance, and easily confused old materials must be clearly labeled with their source.

The desk did not expand endlessly because of this, yet it became like an orderly room. The judge still read quickly, but was still affected by the desktop; the clerk's skill was to let the limited desktop carry the most useful clues. Later, when newcomers were initiated, the old master no longer began with abstract categories, but had them review that failure: where intuition had originally been relied on, why the first repair was not enough, and how the new arrangement kept risk in a place that could be checked. Only when they could explain these details clearly, and also point out when to continue using it and when to escalate, were they considered to truly understand this method. Later they also handed this method to newcomers for practice: first have the newcomer do it the old way and see how errors appeared; then redo it under the new rules and see which step was blocked in advance and which step left evidence. If newcomers could only recite names, they would still misuse it in similar scenarios; only those who could state the applicable boundaries, the costs and the remedies after failure were allowed to take work independently.

Reveal

This story is about: context window

Context Window is the total amount of input and output a model can process in a single request, usually measured in tokens. It determines the space that prompts, retrieved materials, historical messages, tool results, and generated answers can occupy together. Long-context models reduce truncation risk, but they do not automatically solve problems of relevance, ordering, contamination, cost, and attention dilution. Enterprise Agents need to make choices, ranking, compression, and budget management around the context window.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Desk: one context window of the model
  • Off-desk case files: information not included in the current request
  • Write smaller: crude compression or cramming in more material
  • A larger desk: a longer context window
  • Put expiration policies somewhere visible: low-quality context affects output
  • Serving rules: context selection, ranking, and compression strategies
  • Final insight: Context Window is a capacity boundary, not a replacement for context engineering

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

tokenpromptlong-context modelattentiontruncation

Related distinctions

This concept is easily confused with adjacent concepts; it is recommended to continue to the comparison page.

京ICP备2026014272号