← Back to concept explainers

Concept Fable

Top-k routing

Conclusion first:Top-k Routing is a strategy for MoE gating: each token is sent only to the k experts with the highest scores, rather than to all experts. The choice of k affects computation, quality, latency, and expert load; Switch Transformer can be seen as the extreme form with k=1, and a larger k usually improves expressiveness but increases cost.

Top-k Routing · MoE

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

Only invite the first few suitable masters for each piece of iron

The market consultation tent originally ran on a familiar set of methods. When the consultation manager saw fewer patients, he let whoever was free take the next consultation, and rarely made mistakes when seeing patients at the old pace.

Later the matter grew larger. As patients increased, the famous doctor was surrounded so tightly that urging voices lined up from early morning to evening. The newcomers thought this was a quantity problem, solvable by sending more people and adding more hours; the veterans, however, saw that many details had already crossed the boundary that the old methods could bear.

The person in charge tried the first approach: have every physician see every patient. In the first few days there was indeed improvement, the queue got shorter, and outsiders felt this craft had finally caught up with the new times.

But trouble quickly changed shape. Some people skipped inspections to rush progress, some forced unsuitable work into old processes, and the most skilled people were dragged down by trivial matters. Errors were not always visible on the surface and often only emerged after delivery. The old ledger could still explain where things came from, but it could not explain how to change for the next step.

The person in charge added more rules, requiring everyone to sign more, report more, and add another layer of protection. On paper it looked thorough, but on site it was slower; the craftsmen were busy proving they were not wrong, while the real cracks continued to widen in place. Onlookers only saw results fluctuating between good and bad, and people on site found it increasingly difficult to judge whether to save speed, quality, or responsibility first. Everyone felt they had done a little more, yet the whole thing did not become more stable because of it.

A silent old craftsman laid several failures on the table. He blamed no one, only had everyone look at the same detail: the new restriction had never entered daily practice, and only suddenly appeared before delivery. He laid the several reworks out in the order they occurred, and only then did everyone discover that mistakes were always amplified at the same corner.

So the workshop adopted a new rule: the doctor at the door would first select the two or three most suitable specialist physicians to receive patients. At first, everyone was uncomfortable, as if their hands and feet were tied; but after several rounds, rough shortcuts were blocked, and truly reliable actions gradually remained.

When the same pressure came again, delivery did not become fancier, but it was clearly more reliable. The intake supervisor finally understood: sending problems to the few most suitable people saves effort and reduces chaos. If you wait until the end to remedy things, the cost is often already buried in every earlier choice; this new path did not eliminate cost, but made cost appear earlier, making it easier to handle while changes were still possible.

Reveal

This story is about: Top-k routing

Top-k Routing is a strategy for MoE gating: each token is sent only to the k experts with the highest scores, rather than to all experts. The choice of k affects computation, quality, latency, and expert load; Switch Transformer can be seen as the extreme form with k=1, and a larger k usually improves expressiveness but increases cost.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • All masters give opinions: sending tokens to all experts is computationally expensive
  • Invite only the top one: top-1 routing is fast but limited in capacity and robustness
  • Lock-shaped axle: complex tokens requiring multiple expert perspectives
  • Top-k masters: Top-k Routing
  • Set capacity for each job: expert capacity and token dispatch limits
  • Record the effects of different k values: tuning quality, cost, latency and load
  • Final insight: Top-k Routing trades off between sparse computation and expressive capacity using a controllable number of experts

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

expert routingMoESwitch Transformer (k=1)
京ICP备2026014272号