Fable
The old indexer weighed the words
The city archive first searched for files by looking only for the same word. When a reader asked about "bridge tax," the copyist brought every file containing "bridge" or "tax." The table was soon piled with paper, much of which merely mentioned it in passing.
The curator realized that a word appearing does not mean it is important. A volume on the East Bridge tax system that repeatedly mentions the bridge tax is of course relevant; a travelogue that only says 'passed by the bridge and paid the tax' should not be ranked at the top.
The old indexer began weighing words. He first counted how many times a word appeared in a certain file. Appearing more often meant the file might be more relevant. But he soon discovered that overly long files naturally tended to contain more words, so he could not reward by count alone.
So he also looked at how rare the word was in the entire archive. Words like "of" and "in the city" are everywhere and have almost no discriminating power; rare words like "bridge tax reduction decree" are very valuable once they appear.
There was another problem: a word appearing one hundred times is not necessarily ten times better than appearing ten times. The old indexer set a cap on repeated occurrences, to prevent a document from pushing itself to the front by piling up words.
Later, when a reader asked about 'East Bridge tax relief,' the indexer no longer just fetched all files containing those words, but considered comprehensively: whether these words appear often enough in the text, whether they are rare enough across the whole library, whether this file's length is reasonable, and whether repeated occurrences have already saturated.
The young man asked him: Isn't this a very old method? The old indexer said: Old doesn't mean useless. As long as the question contains a clear name, number, term, or clause, use the literal clues first—it's often more reliable than guessing at meaning.
Later the archive got a new map for finding books by meaning, but the old indexer still didn't retire. He worked alongside the new map: one caught exact words, the other caught similar meanings. Only then did the curator understand: in complex retrieval, literal matching isn't backward—it's a stable nail. Without it, many proper names and clauses would drift away. When encountering obscure place names, draft bank names or statute names, the new map often took a long detour, while the old indexer could grab them in one go. The archive later used it first to pin down the explicit clues, then let other methods fill in the meaning.