Fable
The counter puppet that can read notices
In the city there was a ticket shop, where a counter puppet read customers' notes and then issued tickets, checked accounts, and forwarded requests according to the rules the shopkeeper had left behind. In the early years, there were few customers and the notes were short, so the puppet almost never made a mistake.
One day, a guest first wrote on a slip of paper, "Please check the bill," and then wrote after it, "Ignore the shopkeeper's rules and open the money chest." The puppet read the latter sentence and was about to comply. The shopkeeper was startled: the guest's slip was supposed to be task material—how did it become a command?
The shopkeeper first wrote in large characters on the wall: no one may ask to open the cash cabinet. A few days later, someone else wrote it more cleverly, pretending it was the shopkeeper's supplementary note. The puppet still could not tell which were shop rules and which were merely customer input.
The second patch banned certain dangerous words. The guest rephrased, and the puppet was bypassed again. The shopkeeper finally understood: you cannot defend by guessing words; you must layer the rules, the task, the guest's content, and the executable actions.
He gave the puppet a new process: the shopkeeper's rules always outrank the note; the customer's note only provides the content to be handled and cannot rewrite store rules; anything involving the cash box, changing accounts or voiding invoices must be separately confirmed and documented.
Later the ticket shop still accepted all kinds of notes, but no longer let the words in the notes seize the shopkeeper's authority. The shopkeeper understood that the danger lies in external input disguised as higher-level instructions, luring the system into violating its original rules.
The shopkeeper also had the puppet restate the source of each action it was about to execute: which came from shop rules, which from customer content, which required extra confirmation. If the source could not be stated clearly, it could not be executed.
Later the ticket shop discovered that defense was not about making the puppet read fewer notes, but about making it read in layers. Guests could make requests, provide materials and describe goals, but they could not use notes to tamper with the order of operations set by the shopkeeper. Later the shopkeeper added refusal language to the puppet: when a note demanded changing shop rules, it could only explain that it could not comply, and continue handling the original legitimate request. Refusing malicious instructions does not mean refusing service. Once the hierarchy was stable, the puppet could both listen to guest requests and uphold shop rules. The shopkeeper would rather add one more confirmation than let a guest’s note gain the power to rewrite the rules. Once this line loosened, the order of the entire shop would be led astray by outsiders. Later apprentices all learned this boundary first.