← Back to concept explainers

Concept Fable

Cross-encoder

Conclusion first:A Cross-Encoder inputs the query and candidate text together into the same model, letting the model judge relevance in a joint context. It is usually more precise than methods that encode the query and document separately, and is suitable for RAG reranking, search result fine ranking, question-answering evidence selection and compliance material matching. The cost is heavier computation, and it cannot pre-encode all documents offline like vector retrieval, so it is often used for candidate reranking after first-stage retrieval.

Cross-Encoder · Reranking

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The two case files were finally placed on the same table

The city archive had two teams. The first team ran fast, going to the storeroom to haul back a stack of candidate files based on the question; the second team was responsible for preparing the final desk for the director. In the past, the second team looked only at file covers and question summaries, ranking by similar words.

Once the chamber of commerce investigated an old contract. Two file covers both read "Wharf Lease," one a renewal and one an early termination. The fast-running team brought them all over, and the sorting clerk looked only at the covers and put the renewal file first. The curator almost cited the wrong case.

The supervisor first had the cover written in more detail. The archive therefore busied itself relabeling, but it still made mistakes when encountering ambiguous questions. Questions and files were stored separately, and the filing clerks could not see the fine interlocking points between them.

Later the old archivist changed the desk rule: each time candidate files were rearranged, the question and one file were spread on the same desk and compared line by line. If the question was about early termination, he looked for clues such as breach, term, and notice in the file, rather than just shared words.

This was much slower and could not replace the first team's massive book-finding. But when only a few dozen candidates remained, reading closely at the same table could push the most relevant files to the front, and could also reveal materials with similar covers but opposite meanings.

The archive finally settled on a division of labor: first use a fast method to narrow the scope, then use close reading at the same table for fine ranking. The director thus understood that some judgments must let the question and the material see each other; understanding them separately and then comparing often fails to grasp the real relationship.

The old librarian also set timing limits. Reading closely at the same table is labor-intensive; if every item in the entire library were read closely from the start, the librarian would collapse. So it is used only in the final stage, when candidates have already been narrowed down and ranking quality determines success or failure.

Later the curator explained to newcomers: the fast-running team is responsible for not missing possible objects, and the close-reading desk is responsible for not mistaking similar shells for relevant answers. One emphasizes recall, the other emphasizes judgment. Reading the two sets of materials together is costly, but it buys finer relevance. Some files were therefore moved down even though their titles looked very similar; some files were moved up even though their wording differed but they answered the question. What the curator wanted was exactly this kind of ordering after close reading, not the illusion created by similar covers. It is slower, but it reduced misjudgments in the final steps. This step is often the difference between the correct answer and similar-looking misdirection.

Reveal

This story is about: cross-encoders

A Cross-Encoder inputs the query and candidate text together into the same model, letting the model judge relevance in a joint context. It is usually more precise than methods that encode the query and document separately, and is suitable for RAG reranking, search result fine ranking, question-answering evidence selection and compliance material matching. The cost is heavier computation, and it cannot pre-encode all documents offline like vector retrieval, so it is often used for candidate reranking after first-stage retrieval.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • First team: bi-encoder or vector retrieval, responsible for quickly recalling candidates
  • Score questions and files separately: encode independently, then compare by similarity
  • The examiner and the candidate read sentence by sentence side by side: cross-encoders jointly read the query and document
  • Procurement compliance issue: enterprise RAG's need for fine-grained condition matching
  • Only process the candidate set: cross-encoders are often used for reranking, not full-database scanning
  • The last mile: the evidence reranking stage before entering the LLM context
  • Slow within a small scope: the engineering trade-off among accuracy, latency, and cost

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

rerankerbi-encoder

Related distinctions

This concept is easily confused with adjacent concepts; it is recommended to continue to the comparison page.

京ICP备2026014272号