← Back to concept explainers

Concept Fable

Data leakage

Conclusion first:Data leakage means information that should not appear during training, tuning, model selection, or evaluation enters the system early, distorting performance estimates. It includes not only leaked test answers, but also target labels, future information, human annotations, customer outcomes, similar examples, or improper feedback from production. Quality evaluation for enterprise Agents requires clearly separating training, development, regression, and holdout test data, controlling who can see the answers, recording sample provenance, and avoiding launch decisions based on leaked scores.

Data Leakage · Evaluation

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The sealed answer was passed in through the crack in the door

South City Academy holds a herbal medicine exam every year. Students must smell leaves, identify roots and write out medicinal properties, and the master uses this exam to judge who can truly enter an apothecary.

In the first year, exam questions and answers were locked in two separate cabinets. Scores varied, and the teacher could see who had worked hard on ordinary days. In the second year, for convenience, the person managing the papers placed the answer booklet in the grading room in advance, with the door crack directly facing the waiting corridor.

The students didn't openly peek, but caught a few lines while queuing: clover turns red in water, bitter vine root has horizontal stripes. On exam day, many answered beautifully, and the teacher thought teaching had made great progress.

But when they got to the pharmacy internship, these students fell apart the moment they encountered a herb that had changed color. The shopkeeper discovered that they had memorized the hints leaked during exams, not how to truly identify the herbs.

The master's first fix was to cover the answer book with cloth. But the page numbers showed at the corner of the cloth, and students could still guess which herbs would be tested. Blocking only the obvious content cannot block indirect clues.

Later the academy completely separated question-setting, exam preparation, grading and internship materials. Any draft that could appear in the exam hall could not be marked in advance during daily practice; any answer used for judging could not appear in any preparation step.

The next year's scores dropped somewhat, but they were more real. The master finally understood that if an exam is to measure ability, neither the answers nor their shadows may flow into the practice and evaluation process in advance.

The academy later discovered that leakage is not always as obvious as peeking at answers. Sometimes it is the paper color of the question booklet, the order in which herbs are placed, or the examples the teacher repeatedly emphasizes in class. Any clue that lets students guess the evaluation content in advance can contaminate the exam. A truly clean evaluation must separate learning materials from testing materials far enough apart. Later, whenever the teacher reviewed an exam, he checked from the door, the cabinet, the textbook, and the practice questions all the way to the grading rules. As long as evaluation information leaked into the learning process in advance, scores would look better but be distorted. The academy would rather have ugly scores than not know the students' true ability. Later, even when scores dropped, the teacher felt more reassured, because ugly true scores are more valuable than beautiful false progress. This saying was later written into the newcomers' handbook, reminding everyone to look at the on-site cost first before talking about saving effort. And because of this reminder, when similar difficulties arose later, people would not rush to apply old methods.

Reveal

This story is about: data leakage

Data leakage means information that should not appear during training, tuning, model selection, or evaluation enters the system early, distorting performance estimates. It includes not only leaked test answers, but also target labels, future information, human annotations, customer outcomes, similar examples, or improper feedback from production. Quality evaluation for enterprise Agents requires clearly separating training, development, regression, and holdout test data, controlling who can see the answers, recording sample provenance, and avoiding launch decisions based on leaked scores.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Practice cloth: training data or development samples
  • Acceptance cloth: test set, holdout evaluation set, or pre-launch validation set
  • Scraps and labels: fields or metadata that seem harmless but contain answer clues
  • Sudden score improvement: inflated metrics after leakage contamination
  • New customer fabric rework: production generalization capability did not truly improve
  • Sub-warehouses and registration: dataset isolation, access control, and lineage records
  • Blind review: preventing target information from entering the review process
  • Final insight: Data Leakage makes teams mistake test answers for system capability

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

contaminationbenchmark leakagetest setoverfitting
京ICP备2026014272号