Concept Contrast Fable

What is found by vectors and what is found by keywords

Conclusion first:When building a RAG system, you will encounter two different retrieval approaches: one finds documents by "computing similarity," the other by "matching keywords." On the surface both are search, but in practice they are two entirely different logics.

When building a RAG system, you will encounter two different retrieval approaches: one finds documents by "computing similarity," the other by "matching keywords." On the surface both are search, but in practice they are two entirely different logics.

Fable

The two archivists

Two masters are hidden in the ancient town's archive. Old Lin has managed the inner vault for thirty years and has never used an index card. He has people describe what they are looking for, then walks among the shelves with his eyes closed and pulls out exactly the volumes you need. When asked how he does it, he can't explain, only saying, "After flipping through enough, you get a feel for it."

Xiao Zhang is the file clerk who joined last year. He made a keyword card for every file and attached it to the corresponding folder. To find anything related to "contract," he pulled out all the folders whose cards said "contract," not taking one extra word. The operating manual was posted on his desk: when a new customer arrives, check the cards first.

Someone came to look for the land transfer documents. Old Lin thought for a moment and pulled out one scroll about the land deed, one about property rights disputes, and one about the sale contract. Xiao Zhang only pulled out the two volumes whose spines clearly said contract. When the two put the results together, the difference was obvious at a glance.

At first everyone thought Xiao Zhang was reliable—nine out of ten things he found were correct. Of the ten items Old Lin found, two or three always looked relevant, but on closer inspection they were not. The director praised Xiao Zhang at the quarterly meeting, saying the indexing work was solid.

But after a while, people coming to search began to complain: I want to find "the matter of buying land" — why don't you have it? The archivist pointed them to the "property transfer" volume, and only then did they realize — apparently a different wording meant it wasn't there. The same person who last time searched for "home purchase agreement" could not find it this time when they said "buying land."

Old Lin heard the dispute and said quietly: no matter how complete the index is, it can't hold up when the speaker changes words. Word reached the curator, who fell into deep thought. He called both men in and asked whether both methods could be used at once.

Later, the curator came up with a plan: let Xiao Zhang keep guarding the first gate, using keyword cards for a rough screening; let Old Lin guard the second gate, screening again with his instincts. Only what passed both gates was delivered to readers. This avoided both missed terms and off-target files. Later he added another rule: for urgent items, find the doorplate by keyword first; for difficult items, find the familiar expert by scent. No one should try to cover everything.

From then on, the archive had two parallel retrieval logics: one locked down keywords, the other opened up similarity. When to use which door, or both together, became a required course for every new apprentice.

Dense Retrieval (vector retrieval)

Convert both documents and queries into vectors and compute similarity in a high-dimensional space. It can find documents with different wording but similar meaning.

Sparse Retrieval (keyword retrieval)

Use an inverted index to match query terms with terms in documents. Precise but prone to missing related content phrased differently.

Story Mapping

Old Lin's feeling → vector similarity calculation
Xiao Zhang's keyword cards → inverted index matching
The two-door parallel approach → Hybrid Search
Finding the door number by keyword, like sparse retrieval using explicit clues to locate first
Finding acquaintances by scent, like dense retrieval finding semantically similar content
Urgent documents and difficult documents are handled separately, showing that the two retrieval methods suit different problems
The old shopkeeper combined the two methods, corresponding to the actual choice of hybrid retrieval

In the RAG system

If your knowledge base content has fixed phrasing and unified terminology (such as product manuals or contract templates), keyword retrieval is enough. If the content comes from real user questions with varied phrasing (such as customer service records or forum posts), prioritize vector retrieval. A better approach is to combine both: first use keywords to filter the candidate set, then use vectors to rerank, balancing recall and precision.

What is Dense Retrieval What is Sparse Retrieval