← Back to concept explainers

Concept Fable

TruLens

Conclusion first:TruLens is an LLM application evaluation and observability library, commonly used to add feedback functions to RAG and Agent applications to measure quality signals such as relevance, groundedness, and toxicity. Its core value is not giving a vague score, but breaking answer quality into explainable, traceable, regression-testable evaluation dimensions.

TruLens · LLM evaluation and observability library

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

Three mirrors revealed what was real and what was not in the answers

There was a Q&A teahouse in the city, where guests came with ledgers, prescriptions, and contracts to consult the proprietor. At first the proprietor answered from experience, the guest nodded, and the teahouse counted it as a completed order.

As the business grew, apprentices began helping by checking files before answering. The answers became smoother and smoother, but smoothness does not equal reliability; some replies were unrelated to the question, and some sounded reasonable but had no evidence.

The shopkeeper first required apprentices to write a few more sentences of explanation when answering. Once the explanations grew longer, it became harder for customers to spot errors, and apprentices became better at making unchecked content sound as if it had been checked.

Once a merchant signed a bad contract on the teahouse's advice. In the retrospective, everyone argued only about whether the answer looked like something an expert would write, but no one could separately judge whether it answered the question, whether it was based on the materials, or whether it smuggled in dangerous advice.

An old tea master brought out three mirrors. The first checked whether the question and answer were relevant; the second checked whether the answer was supported by the case files; the third checked whether the materials found were actually useful.

He had each apprentice look in the mirror after answering. The mirror wouldn't answer for the apprentice, but it would leave scores, reasons and failure locations, letting the shopkeeper know whether to fix the grading, the questioning method, or the final expression.

Later, the teahouse also turned toxic advice, off-topic questions, and unsupported assertions into checklist items. Different businesses used different combinations of mirrors—you can't let one ruler pretend to fit every scenario.

Before new apprentices entered the hall, they first had to answer old customers' questions and pass the mirror inspection; after launch, customer complaint samples continued to flow back and became the next round of tea evaluation material. Only then did the shopkeeper understand that complex Q&A cannot just ask whether the answer looks beautiful; it must also reflect where it came from, whether it hit the point, and whether it might harm someone. The old tea master also required that mirror inspection results feed back into the process. If relevance was poor, change how questions are routed; if case-file support was weak, change how files are searched; if there was dangerous advice, add interception and review. Looking in the mirror was not for assigning scores, but for knowing where to make the next cut. The teahouse later hung the mirrors behind the counter, where apprentices could see them every day. They reminded everyone that smoothness is only surface; only answers that can be held up to questions, materials, and safety boundaries are worth giving to customers. Later, during the review, everyone wrote this lesson into daily rules: first look at where the scene actually changed the outcome, then decide what to keep, what to fix, and what to restrict.

Reveal

This story is about: TruLens

TruLens is an LLM application evaluation and observability library, commonly used to add feedback functions to RAG and Agent applications to measure quality signals such as relevance, groundedness, and toxicity. Its core value is not giving a vague score, but breaking answer quality into explainable, traceable, regression-testable evaluation dimensions.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Q&A Teahouse: enterprise knowledge Q&A, RAG or Agent applications
  • Longer explanation: a fix that uses surface fluency to cover up quality problems
  • Three Mirrors: TruLens Feedback Functions and the RAG Triad Approach
  • Relevance: answer relevance or context relevance
  • Dossier support: groundedness / faithfulness
  • Toxic advice check: toxicity, safety, and compliance scoring
  • Customer complaint sample feedback: production failure cases enter regression evaluation
  • Final insight: TruLens breaks LLM quality judgment into executable, explainable evaluation signals

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

feedback functionevaluationgroundednessRAG triadobservabilitytraces
京ICP备2026014272号