Fable
Several messenger teams share one key room
The royal city's reply office originally assigned a key room to every messenger team. Each team could check old letters, retrieve indexes, and write replies quickly, but the warehouse grew more crowded and the key administrators multiplied.
The accountant discovered that what was truly expensive wasn't the messenger writing letters, but saving and hauling large indexes of old letters at every step. The more long letters there were, the more space the key room took up.
The director first cut part of the messenger team. Costs went down, but reply quality dropped too, because different teams had originally been responsible for different perspectives.
The second repair was to truncate old messages. It was faster, but as soon as long conversations, long code, or multi-turn tool results appeared, key context was lost.
Later they divided the messenger team into groups. Within a group, multiple query teams still kept different observation angles, but shared the same set of keys and value store, reducing repeated access.
This is not a complete return to one big key room, nor does each team monopolize one. It takes a middle path between expressive capability and reasoning efficiency.
When enterprises deploy giant scribes, this difference is very real: the longer the context and the higher the concurrency, the more likely KV Cache usage becomes a cost bottleneck. Grouping can reduce memory pressure while preserving multi-head query capability as much as possible.
The director finally said that saving money cannot mean only cutting capabilities, nor can every team be allowed to duplicate everything. A good architecture preserves what must be different and consolidates what can be shared. Later, an even trickier job came along. On the surface it looked like just the old problem magnified, but in fact it forced out all the gaps hidden in the process: who takes over first, who can make changes, and how to return to the previous step when something goes wrong. Places that used to get by on familiarity and tacit understanding now all had to be made explicit. The person in charge did not urge everyone to be more careful again; instead, he walked the site once more. He had people record every handoff, every failure, and every remediation, noting which places were merely slow, which places could harm the results, and which places, once wrong, would be hard to trace back. After the new rules were rolled out, the first few days did not go smoothly. Some complained about an extra check; some felt their old experience had been insulted. But when the next anomaly appeared, for the first time everyone could follow the traces to find where the problem had occurred, instead of gathering around the result and guessing at one another.