Concept Contrast Fable

Context engineering and retrieval-augmented generation

Conclusion first:RAG tells you “how to feed external knowledge to the model”; context engineering tells you “how good the fed-in content is, whether it is well ordered, and whether it has overflowed the window.” Discussing only RAG without context engineering will most likely stuff a poor retrieval result unchanged into the model.

RAG tells you “how to feed external knowledge to the model”; context engineering tells you “how good the fed-in content is, whether it is well ordered, and whether it has overflowed the window.” Discussing only RAG without context engineering will most likely stuff a poor retrieval result unchanged into the model.

Fable

Two processes in the archive

In the east of the city stood an archive holding hundreds of thousands of files. The curator designed a process: when someone came to look up materials, the clerk would first search the stacks for relevant files, pick out the twenty most similar, carry them to the reading room table, and ask the researcher to leaf through them personally. This process ran for over a year, and the curator named it "retrieval-assisted reading."

Problems gradually appeared. Some people looking up information reported that among the twenty scrolls on the table, only five or six were truly useful; the remaining dozen or so were either outdated or about something else. More troublesome was that the twenty scrolls already filled the table, and when they wanted to see more related information, there was no place to put it.

The curator brought in an organizer. The organizer did not touch the retrieval step—she did not care how the files in the archive were selected. She was responsible for only one thing: after the twenty retrieved volumes arrived in the reading room, how to organize them, how to order them, how to choose among them.

The organizer added three operations on the reading-room table. The first was called filtering: of the twenty scrolls, the obviously irrelevant ones were removed first, freeing up table space. The second was called sorting: the remaining scrolls were rearranged by relevance to the query, with the most relevant on top. The third was called summarizing: for especially long scrolls, key passages were marked so that the researcher could find the main points at a glance.

Just adding these three steps improved the feedback from people looking up information a lot. The desktop was no longer piled with useless information, and answer quality improved noticeably. But interestingly, the director had always blamed "inaccurate information retrieval" on the retrieval system not being good enough, repeatedly tuning retrieval parameters and switching indexing schemes, with limited results. After the organizer arrived, the same retrieval results jumped up a level in quality.

The curator then formally split the process into two stages: the first was called "retrieval," focused on finding the relevant files from the archive. The second was called "context organization," focused on organizing, ranking, truncating, and annotating to ensure that the information actually delivered to the researcher's eyes was refined.

Later, when the archive introduced itself to the outside world, it no longer said only "we have retrieval-assisted reading." The curator would add: retrieval is the first step, context organization is the second. Skip the second step, and even the best retrieval results may be wasted.

A peer came to learn from them and asked the curator what the archive's core competitiveness was. The curator said: everyone uses similar retrieval technology; what sets us apart is the context organization step. Whoever keeps the desk clean provides better service.

Context Engineering

An engineering practice for systematically managing and optimizing "what the model ultimately sees." It covers post-retrieval filtering, reranking, truncation, compression, deduplication and formatting, with the goal of maximizing information density within a limited context window.

Retrieval-Augmented Generation (RAG)

An architectural pattern that combines external knowledge retrieval with language model generation. The core chain is: query → retrieve → insert results into the prompt → model generates an answer. It defines the overall framework for "how to feed knowledge in".

Story Mapping

The retrieval system selects twenty volumes from the library: the retrieval step of RAG.
The organizer's three operations (filter, rank, summarize): the core actions of context engineering.
The curator repeatedly adjusted retrieval parameters but the effect was limited: optimizing only at the RAG layer while ignoring context engineering.
Desktop space is limited: a model's context window has a length limit, and invalid content crowds out the space for valid information.
What sets systems apart is context organization, not retrieval technique: the bottleneck in current RAG systems often lies after retrieval.

In the project

When evaluating the effectiveness of a RAG system, measure retrieval quality and context quality separately. Retrieval quality looks at hit rate and recall, while context quality looks at the information density, relevance ranking, and truncation loss of what is ultimately stuffed into the prompt. A common analytical signal: if retrieval hit rate is high but final answer quality is poor, the bottleneck is most likely in context engineering—for example, an unreasonable reranking strategy, or key information being cut off during truncation.

Context Engineering Retrieval-Augmented Generation