Concept Fable

MoD

Conclusion first:Mixture of Depths dynamically decides whether different tokens need to pass through all transformer layers, letting some tokens skip unnecessary layers and thereby reducing computation. It belongs to the dynamic computation approach, adjacent to efficiency techniques such as early exiting and MoE. The core difficulty is determining which tokens can be computed less without harming overall quality.

Mixture of Depths · Architecture

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

Not every plank has to go through the twelfth planer

The Wooden Bridge Workshop had twelve planing stations. Whether the board was a thick beam or a small shim, it had to go from the first station to the twelfth, and the old rule said this was the fairest way. When there was little wood, no one felt it was wasteful.

Later, Qiaofang worked on beams, handrails, wedges, and temporary shims at the same time. A good wooden beam really does need layer after layer of fine planing, but it makes no sense for a small shim to go through all twelve steps; some rough beams were found to have cracks at the third step and were still sent for fine finishing, needlessly occupying the stations behind them.

The workshop owner first added a planer so every board still went through the full process. The yard got more crowded, waits got longer, and simple wood pieces occupied the fine workstations that should have gone to complex beams. A fair process actually slowed down the critical components.

The old carpenter proposed setting checkpoints after every few process steps. If the board was already flat enough, it went straight out; obvious cracks were sent to repair or the scrap pile; only beams and handrails that needed detail continued into the deep planer.

At first, the apprentice worried that early release would lead to cut corners. So the old carpenter required every checkpoint to leave a reason for the judgment and a minimum standard. Saving work was not skipping at random but deciding by material condition whether further processing was still needed.

The bridge workshop later became faster and more stable. Complex timber received more deep processing, and simple wood chips no longer exhausted all the processes. The workshop owner understood that not every input needs the same depth; letting different materials go through different numbers of layers is how capability is spent where it is truly needed.

The checkpoint later learned to watch the queue too. If the fine planer was congested, ordinary shims went out earlier; if beam quality fluctuated, more beam material went through the deep layer. Depth was not a fixed entitlement, but an allocation based on material condition and resource pressure.

The bridge workshop thus eliminated two kinds of waste: over-processing simple planks and under-processing complex beams. The master wrote this into the workshop rules: give every piece of material enough treatment, and do not force all materials down the same deepest path. Later the master saw output rise without lowering acceptance standards. Planks finished early still had to be measured; deep-processed beams still had to be checked for load-bearing. Saving depth must not sacrifice results—only reduce unnecessary paths. This kept deep processing from being spread evenly and reserved it for the materials that were truly hard to handle. Later the apprentice understood: fairness is not every piece of wood taking the same path, but every piece of wood receiving suitable treatment.

Reveal

This story is about: MoD

Mixture of Depths dynamically decides whether different tokens need to pass through all transformer layers, letting some tokens skip unnecessary layers and thereby reducing computation. It belongs to the dynamic computation approach, adjacent to efficiency techniques such as early exiting and MoE. The core difficulty is determining which tokens can be computed less without harming overall quality.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

Twelve planers: the Transformer’s multi-layer network
Every plank takes the full route: every token passes through all layers
Simple wood chips occupy the deep planer: low-difficulty tokens consume unnecessary computation
Entrance check ruler and mid-course re-judgment: dynamic routing or depth selection
Early Exiting: Skip later layers or reduce computation depth
Continue processing dark-crack beam material: complex tokens use more layers
Final insight: Mixture of Depths turns computational depth into an on-demand resource

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

early exitingMoEdynamic computationefficiency