Fable
The port dispatcher added invisible weight to the cargo tags
The river port had to handle scheduling for thousands of boats every night. The old dispatcher relied on a thick cargo tag register: which dock each boat berthed at, which cargo owner it belonged to, what it carried, and where it was headed.
The problem began with "find similar goods." A caravan needed a batch of cotton cloth close in quality to the last one, but the tag said only "cotton cloth, grade two." The dispatcher searched for a long time and found that ships also labeled "cotton cloth, grade two" felt very different because of origin, thread count, and dyeing method.
Someone introduced a new label: add a ciphertext description to each type of goods, and the ciphertext descriptions can be directly compared for similarity. On the first day the tool went live, the caravan was overjoyed, and the dispatcher breathed a sigh of relief.
Before long a new problem appeared. The caravan not only looked for similar cotton cloth, but also added constraints: "close to the last batch, but made in Shandong, priced within three taels, arriving within the last three days." Ciphertext can compare similarity, but it cannot compare origin, price and arrival date.
The dispatcher had no choice but to first pull out all ships matching "Shandong origin, within three taels, arriving in three days," then compute ciphertext similarity one by one across those hundreds of ships. Both steps were slow, and together even slower.
They switched to a new scheduling system. Each ship's tag records both ordinary fields and a description ciphertext. At query time, an exact filter runs first on the ordinary fields to narrow the matching range; then, within that range, similarity search runs only on the ciphertext. Both operations complete in a single query, with no need to run them separately.
With this system, dispatchers can run more complex hybrid queries: "first filter the boats from Jiangnan last December, then find the five most similar to this batch among the rest." Filters run on structured fields, similarity runs on ciphertext, indexing and computation are independent, and results come back in one step. The new system also lets trading caravans see only their own cargo domain. Caravan A cannot find Caravan B's private goods, and ordinary dispatchers cannot freely browse high-priced contracts. By putting similarity search and precise restrictions together, speed, permissions and updates all become manageable at the same time.
Later the system was upgraded to track query speed, disk usage and the update frequency of each cargo tag at the same time. The dispatch desk could even divide multiple subsets within the same dataset, so that different caravans saw only the shippers they had contracted with. The old dispatcher put the register back in the drawer: a truly useful dispatch system must know not only how similar goods are to each other, but also whether goods can be shipped and whether they should be.