← Back to Concept Contrast

Concept Contrast Fable

Model routing and Agent routing: one sends the question to the right model, the other assigns the task to the right role

Conclusion first:Model routing decides at the inference level which model a request should go to—chosen by cost, speed, or capability. Agent routing decides at the task level which Agent an intent should be assigned to—chosen by domain responsibility and permissions. Both are called routing: one does traffic distribution inside the inference engine, the other does role assignment at the business orchestration layer.

Model routing decides at the inference level which model a request should go to—chosen by cost, speed, or capability. Agent routing decides at the task level which Agent an intent should be assigned to—chosen by domain responsibility and permissions. Both are called routing: one does traffic distribution inside the inference engine, the other does role assignment at the business orchestration layer.

Fable

There are two kinds of sorters in the government office: one looks at envelopes, one looks at cases

The Ministry of Revenue received documents from all directions every day: tax contracts, disaster relief grain, frontier military supplies, and commercial travel notices. At first all documents were sent to the same copying room and cleaned up by the same batch of clerks. Short messages were fine, but when accounts, legal statutes, and terrain maps were mixed together, the copying room often copied the characters correctly while getting the matter wrong.

The Minister first changed the transcription channels. Documents with messy handwriting went to clerks who could read cursive script, those with many counting rods went to accounting clerks, and foreign documents went to translation clerks. This step made each document easier to understand, but it did not decide who would inspect the granaries, who would send letters, or who would seek instructions.

One urgent disaster-relief document was copied very accurately by the scribe, yet no one dispatched grain; another tax deed was sent to the accountant who calculated the silver clearly, yet no one had the inspector verify the land boundaries. Only then did the minister understand: choosing someone who can read a certain type of document solves only the reading problem, not the question of who is responsible for acting.

Later the Ministry of Revenue set up separate bureau plaques. Documents first chose a copying room by difficulty, and after fair copying were handed by matter to the grain bureau, tax bureau, post bureau, or penal affairs room. Each sub-bureau had its own authority, time limits, and receipts.

The minister initially assigned both matters to the copying office. The copying office quickly became overwhelmed: clerks who could read cursive began deciding the order of disaster relief grain, ledger clerks began sending letters on behalf of the criminal affairs office, and translation clerks, because they knew foreign languages, were asked to receive merchants and travelers. Documents were read faster, but responsibilities became increasingly blurred.

A single border-supply matter involved grain, post horses and armaments, and the copy office could only judge which pages were hard to read—it could not decide which bureaus should join forces, who had authority to open the granary, or who was responsible for reporting back. Only then did the minister separate "understanding" and "assigning duties" onto different boards: the former selected people by material difficulty, the latter assigned roles by functional responsibility.

Later, when new clerks entered the office, the minister always had them look at two boards first. Difficult material goes to those who can read it; heavy cases go to those who can take responsibility. If only one board were set up, the office would sooner or later mistake a clever reader for an official with the power to act.

From then on, the government office had two assignments: one asked "who should read this material," and the other asked "who should handle this errand." If the two were mixed together, the more clearly the characters were copied, the more the work was delayed.

Model routing

During inference, requests are dynamically routed to the most suitable model based on request characteristics (complexity, domain, budget). The focus is on latency, cost, throughput, and matching model capabilities. This is a decision inside the inference engine.

Agent routing

At the task orchestration layer, work is assigned to the most suitable Agent or role based on user intent. The focus is on task decomposition, domain matching, permission control and completion quality. This is a business orchestration layer decision.

Story Mapping

  • Bottom-layer replica preference: model routing—a purely technical choice that does not look at case content
  • The middle-layer old clerk sorts items: intent classification—routing to different domains by business content
  • The top-level deputy hall official defines participating roles: Agent routing—assigning specific Agents and permissions
  • All three layers are called allocation but manage different things: the system design problem of same name, different layers
  • The inspector pressed further: naming distinctions make each layer's optimization direction clear
  • Cross-company co-signing case: the features that require an Agent—task decomposition and permission management
  • Two independent configuration files and a monitoring dashboard: the correct engineering architecture

Applying this to Agent system architecture design

When building a multi-Agent system, draw model routing and Agent routing at two different layers on the architecture diagram. Model routing goes in the inference gateway layer, with configuration items being the model list, latency thresholds, cost budgets, and fallback strategies. Agent routing goes in the orchestration layer, with configuration items being the Agent skill registry, domain matching rules, co-signing policies, and permission matrix. When debugging, if a request went to the right Agent but the returned quality is poor, check the Agent routing configuration; if the Agent's inference is slow or costs are abnormal, check the model routing configuration and model allocation strategy. Both routes share the word "routing," but their configuration files and monitoring dimensions must be maintained separately.

京ICP备2026014272号