Fable
The iron abacus that gives a different verdict when you change the question
The money house had an iron abacus that calculated accounts quickly. When the shopkeeper asked it, "Can this account be approved for a loan?" it gave cautious advice; when a clerk asked, "Help the customer find a reason to pass," it became much more lenient.
At first everyone thought this was the iron abacus understanding human feelings. Later they discovered that with the same material, as long as the question order, tone, examples, or field names changed slightly, the conclusion would differ noticeably.
The staff first standardized the opening, requiring "please be objective and accurate" every time. In the short term it looked a bit more stable, but as long as the customer background was placed before or after, the risk score still fluctuated.
In one approval, Clerk A submitted according to the old template, and the system recommended rejection; Clerk B changed the wording, and the system recommended approval after supplementing documents. The two inputs had the same facts, but the business received two different actions.
The shopkeeper realized that the problem was not a single wrong output, but that the system was overly sensitive to wording perturbations. Such a tool could not support an auditable, scalable process.
He had the accountant establish prompt stability testing. The same case would generate multiple equivalent phrasings, field orders, formats, and example combinations, to observe whether the output maintained the same business judgment. Later, when new clerks joined, the first thing they learned was not how to ask more cleverly, but how to make the same fact return to the same set of rules under different phrasings. Language can be polite, but judgment must not sway with the wind.
For sensitive tasks, the money house switched to structured inputs, fixed templates, clear scoring rules and regression tests. When necessary, the Iron Abacus was only allowed to explain reasons, not directly decide on loans. The accounting office also listed the most volatile words: urgent, help out, as much as possible, for reference only. These words were harmless in themselves, yet they could make the Iron Abacus mistakenly think the business standard had changed.
The iron abacus was not completely unaffected by wording, but fluctuations in key decisions were monitored. Every time the prompt was changed, they had to check whether stability declined. The shopkeeper said: If changing one phrase changes the result, it means we have not yet freed business standards from the surface of language. Later, the manager wrote this experience into daily rules: first look at the scenario, then look at the boundaries, and finally look at who can take over when it fails. Only when all of these are clear will a beautiful demo become deliverable work.