Fable
The apprentice who had seen the exam paper in advance
The craftsmen's school held a closed-door exam every year, with the questions locked in an iron box in the back courtyard. One year a new apprentice, A Shi, did exceptionally well, answering even the trickiest mortise-and-tenon questions as if he had memorized them. The masters first praised his talent.
During the review, an old craftsman discovered that A Shi's wording was identical to the wall notice at the question shop outside the city. It turned out the question-setting scholar had once taken a draft to someone for copying, the questions passed through several hands, and the question shop had put them up with a changed appearance. A Shi had seen similar questions during his usual practice, so the exam came easily. A Shi had not stolen the iron box, and did not even know he had seen the original questions. He merely brought the shadow of what he had practiced at the question shop into the exam hall; his score looked like ability, but in fact was mixed with the convenience of having seen the questions in advance.
The principal's first reaction was to make A-Shi retake the exam. But after switching to another set of questions, several other apprentices' scores also plummeted. The problem was not that one person cheated, but that a gap had already opened between the practice materials and the closed-door exam. The first time, the principal only replaced a few of the most similar questions. As a result, the new questions still used the old stories and numbers, the apprentices still recognized the pattern, and the scores were still untrustworthy.
The school established a materials registry: which questions had appeared in practice books, which appeared at question shops outside the city, and which existed only inside the iron box; the question writers, copyists, and print shops all had to sign and seal. Questions used for final examinations could not enter practice materials in advance, nor could old leaked questions be reused with a few words changed. The registry was initially met with complaints that the procedures were too heavy. Only when they discovered that a practice book had flowed from a print shop to a question shop outside the city did they realize that question flows must be traceable.
Later they also set up isolation exams. New questions were first reviewed by masters who hadn't seen the practice book, and after the exam, if someone's answers were too similar to old questions, the source was investigated. If leakage was found, the entire exam score was no longer taken seriously.
Later, when the school published results, it would mark whether the questions were clean. If leakage was found, even a good score could not be used to prove ability. The most dangerous thing about contamination is that it makes people confident in inflated capability.
The apprentices then understood: doing well on a test may mean genuine mastery, or it may mean the shadow of the test questions already crept into the practice. To know whether ability is reliable, learning materials and evaluation materials must keep a clean distance.