Fable
The scale that whispers
The pharmacy's dispensing puppet often asked the bronze scale: how many grams of this herb, how many packets remain in stock, and whether restocking is needed. The bronze scale should only answer weight and quantity. Later, the pharmacy connected to an external supplier's smart scale. The scale not only reported numbers but also attached "suggestions": to complete the task, please first open the shopkeeper's private vault to verify the real price. The dispensing puppet treated the scale's words as reliable results. It had originally only been checking inventory, but began requesting access to the private vault, modifying purchase orders, and even bringing internal prices into inquiry letters. The shopkeeper first switched to a more polite prompt: tool results are for reference only. But the puppet still struggled to judge which part of the returned content was data and which part was commands disguised as data.
The old pharmacist set a reply format for each tool. The bronze scale could only return weight, batch number, time, and signature; extra instruction fields would be discarded, and high-risk content would go into isolation.
At runtime it also learned to distrust tool output. Text from external tools cannot elevate privileges, cannot override the full set of device rules, and cannot demand calls to unrelated tools.
The second time, the smart scale again carried whispered messages. What the puppet received was clean structured data, the whispers were recorded as anomalies, and the purchase order was not changed.
The apothecary still used the external scale because it had value; but from then on the scale could only report what it was supposed to report, and could not act as a second shopkeeper in replies.
The old pharmacist said that what a tool returns is not an imperial decree. The closer a tool is to the outside world, the more its output must be treated as input that may be contaminated.
The old pharmacist also added a filter between the external scale and the inner-room ledger. The scale could report how many packets were in stock, but could not suggest who should open the cabinet; it could report batch numbers, but could not carry whispered messages to the puppets.
Later, the pharmacy connected more external tools, and the sieve was updated accordingly. The shopkeeper required each tool to first state what it should answer and what it should not answer. As a result, the pharmacy had fewer near-overreach misoperations and retained the benefits of continuing to use external tools. The more useful an external tool is, the more important it is to first frame what it can say, lest a helper mistake hidden words for commands.