← Back to concept explainers

Concept Fable

Draft model

Conclusion first:A draft model is a smaller or faster model in speculative decoding, used to first propose a string of candidate tokens. The larger target model then verifies these candidates and accepts only the prefixes that match its own distribution.

Draft Model · Serving optimization

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The fast-writing apprentice submits drafts first; the old clerk only stamps what can be stamped

The county office had an old clerk who wrote verdicts steadily, but only one sentence each time. As case files piled up, people lined up outside the courtyard. Everyone knew he was accurate, and everyone knew he was slow.

The magistrate brought in a fast-writing apprentice. The apprentice was not responsible for the final verdict; he only sat nearby and, based on the case, first wrote a few possible follow-up passages and handed them to the senior clerk to review. At first, the magistrate had the apprentice write as much as possible. Drafts covered the desk, the senior clerk could not keep up with checking them, and many drafts had gone off track from the very first line, so the whole page had to be discarded.

Later they set the rules: the apprentice had to be fast, the style had to be close to the old scribe's, and the draft length had to be moderate; the old scribe checked one paragraph at a time and accepted as many sentences as he could. When the apprentice guessed accurately, the old scribe seemed to skip several steps; when the apprentice guessed inaccurately, the old scribe continued writing personally from the first wrong character, and the final judgment was still based on his own assessment.

The accountant found that too weak an apprentice wasted verification effort, while too large a one saved no cost. The best fit was cheap, fast and similar enough, but not as strong as the old clerk.

The county yamen also recorded acceptance rates for each type of case. Draft documents with fixed formats were often accepted, while complex disputes rarely passed, showing that not all requests are suitable to be run by apprentices first.

The old secretary then set rules for the apprentice: keep drafts shorter, and guess the most likely few sentences first; as soon as the old secretary saw that the first sentence was off, he would drop the whole passage and stop forcing revisions. The apprentice also learned to observe the old secretary's brushwork, laying out common case types in advance and leaving blanks where misjudgment was easy. The team moved faster, but the old secretary still made the final stroke on the verdict.

This small incident made everyone stop and review. They did not rush to swap in a more impressive-sounding explanation, but instead wrote down clearly, one by one, the entry points that went wrong, the rules that were misunderstood, and the boundaries that needed to be preserved.

The county office later understood that the value of the fast-writing apprentice was not in judging cases for the old clerk, but in submitting in advance candidate sentences that could be quickly adopted or discarded. The magistrate finally understood that the drafter's duty is not to replace authority, but to cheaply explore the next possible segment before authority confirms it, so that expensive judgment walks fewer redundant steps.

Reveal

This story is about: draft models

A draft model is a smaller or faster model in speculative decoding, used to first propose a string of candidate tokens. The larger target model then verifies these candidates and accepts only the prefixes that match its own distribution.

The draft model must be fast enough and close enough to the target model. If candidates are often rejected, verification overhead cancels out the speedup; if the acceptance rate is high, overall decoding speed may improve markedly.

Metaphor mapping

  • Fast-writing apprentice: draft model
  • Old secretary: target model
  • Draft continuation: candidate tokens
  • Old Secretary stamps: verification and acceptance
  • Acceptance rate: acceptance rate in speculative decoding
  • Too weak or too large is not worthwhile: draft model quality-speed tradeoff

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

speculative decoding

Related distinctions

This concept is easily confused with adjacent concepts; it is recommended to continue to the comparison page.

京ICP备2026014272号