← Back to concept explainers

Concept Fable

Faithfulness

Conclusion first:Factual consistency, or Faithfulness, measures whether a model’s answer is faithful to the given evidence or context. An answer may be fluent, logical, and even broadly correct in direction, but if it adds content unsupported by the evidence, rewrites factual relationships, or exaggerates conclusions, it is not faithful enough. For RAG systems, Faithfulness is a key metric for preventing hallucinations and divergence from evidence.

Faithfulness · RAG evaluation

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The stele copyist wrote one extra victory

In Stele Town there was a stele copyist whose handwriting was beautiful. The old steles in the town had weathered, so the county asked him to copy the inscriptions into new volumes for later generations to consult.

There is a stele recording a caravan crossing the snow ridge. On the stele it says: the caravan encountered snow, turned back thirty li, and later changed route and arrived. The person copying the stele felt this was not inspiring enough.

In the new record he wrote: the caravan did not retreat when it met snow, and finally crossed the snow ridge. The new record read with more momentum, and the county magistrate at first even praised his writing.

It was not until descendants of the caravan came to check their ancestors' route that they found the new register did not match the old letters at home. The stele copyist explained that the meaning was about the same: the caravan had arrived in the end.

The old historian shook his head. The stele spoke of turning back and changing course, but the new record wrote it as a successful forced breakthrough. What was wrong was not a small literal error, but that the record departed from the evidence.

From then on, whenever the copyist wrote a line, he marked beside it which stele and which row it came from. What was not on the stele could be written separately as conjecture, but could not be passed off as stele text. The new volume was less dramatic, but more trustworthy, because readers knew every sentence could be traced back to the original stone.

The stele copyist at first felt this way of writing was too constrained. The old historian took him to see scenes of later people checking roads, repairing shrines, and disputing field boundaries, and only then did he understand that the new register would be used by people to make decisions.

If the new booklet reads more smoothly than the old stele but leads people down the wrong mountain path, then the pretty words are harmful. The stele copyist's duty is not to make the story sound more like a victory, but to let readers know what was actually left on the stone. Later, when newcomers started training, the old master no longer began with abstract categories. Instead he had them review that failure: where they had relied on intuition, why the first patch was not enough, and how the new arrangement kept risk in a place that could be checked. Only when they could explain these details clearly, and also point out when to keep using the old approach and when to upgrade it, were they considered to truly understand this method. Later they also had newcomers rehearse with this method: first let newcomers do it the old way and see how errors appeared; then redo it under the new rules and see which step was blocked in advance and which step left evidence. If newcomers could only recite the names, they would still misuse it in similar scenarios; only those who could state the applicable boundaries, the costs, and the remedies after failure were allowed to take on work independently. This checkpoint cannot be skipped. It must also be verified on site. Then handed to someone else for review. Only then can you rest easy.

Reveal

This story is about: faithfulness

Factual consistency, or Faithfulness, measures whether a model’s answer is faithful to the given evidence or context. An answer may be fluent, logical, and even broadly correct in direction, but if it adds content unsupported by the evidence, rewrites factual relationships, or exaggerates conclusions, it is not faithful enough. For RAG systems, Faithfulness is a key metric for preventing hallucinations and divergence from evidence.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Old stele: retrieved evidence, documents or context
  • The stele copyist: a model that generates answers based on evidence
  • One more victory written: the model added facts that the evidence did not support
  • It reads with more force: fluent or more engaging answers do not mean factual consistency
  • Turning back and changing route was written up as a successful forced breakthrough: factual relationships were altered
  • Each sentence is marked with its inscription source: citation, evidence alignment, and traceable generation
  • Speculation must be labeled as speculation: distinguish evidence-supported content from model inference
  • Make every sentence stand: Faithfulness requires answers to be constrained by evidence

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

hallucinationgroundingcitation
京ICP备2026014272号