Fable
The ferry no longer lets every nod automatically launch a boat
A ticket inspection booth was set up at the city gate, where guards judged authenticity by the ticket pattern. At first, the gate captain acted on his inner degree of confidence; with familiar people and familiar routes, he rarely made mistakes when the days were small. The gate captain discovered that the confidence score given by the model was not the conclusion itself; it had to be calibrated before deciding which requests could be automatically released.
Later, during festivals with many people, several old tickets were badly worn, and the guards, afraid of being held responsible, let them all through. As traffic grew, ambiguous tickets brought both wrongful passage and wrongful blocking at the same time; without thresholds set by risk, a single "pass/reject" could not control the cost.
The first fix was rushed: later it was changed so that anyone with the slightest suspicion was stopped, and the city gate formed long lines. Stopping anyone with the slightest suspicion did reduce missed releases, but pushed the cost of false stops and manual queuing onto all genuine visitors. Yet new problems followed: everyone watched only the most visible places, and small deviations in the distance accumulated deeper and deeper.
They tiered by confidence and business risk: high scores are handled automatically, low scores are rejected, the middle zone goes to human review, and real outcomes are continuously used to calibrate the thresholds.
The gate captain had guards sort into three tiers by confidence: very solid ones released, obviously fake ones held back, the middle ones sent aside for re-verification. This step was troublesome and initially slowed the work; but after several rounds, rework decreased, and the masters no longer changed rules on a whim. The guards finally triaged uncertainty, and both automation speed and misjudgment cost fell within an acceptable range.
The point of a threshold is not to chase a pretty score, but to turn uncertainty into executable paths for release, review, degradation, and escalation.
A few days later, there were fewer fake tickets, and real customers were no longer blocked outside the door for so long. From then on, judgment not only had to give an answer, but also had to know to what degree of confidence it deserved to act directly.