← Back to concept explainers

Concept Fable

Top-p sampling

Conclusion first:Top-p sampling, also called nucleus sampling, selects the smallest candidate set whose cumulative probability reaches p, then samples from it. It adapts better than top-k to different distribution shapes: deterministic tasks have few candidates, open-ended tasks have many. Enterprise agents often use it to control diversity, but it does not solve factuality—it needs to be paired with grounding, citations, tool results and safety filtering.

Top-p Sampling, Nucleus Sampling · Decoding

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

Seal the crate only once you have ninety percent confidence

Before going on stage, the storyteller picks the next sentence from a bag of words. If they always pick the most common words, the story is stable but dull; if all obscure words have an equal chance, the audience is often tripped up by strange sentences.

The young storyteller tried selecting words by a fixed number: only looking at the first ten each time. But sometimes good words clustered together, and ten was too many; sometimes the scene was broad, and ten was too few. A fixed basket does not understand the current situation.

The old gentleman changed his approach: first put the most likely next words into the basket in order, until together they covered most of the possibilities, then draw from the basket. In a clear scenario, the basket naturally became small; in an open scenario, the basket became larger.

Once, the apprentice set the coverage line too low, and the story became rigid; too high, and strange words crept in. So he adjusted the line according to the nature of the play: notices should be steady, legends can be loose.

The old performer had him look at the scene first, then choose words. When telling of the royal palace, the common words were already enough to carry the plot, so the basket could be small; when telling of sea monsters and foreign ports, the common words were too narrow, so more possible words had to be put in. The size of the basket was not fixed; it depended on whether the preceding words together already covered most of the possibilities.

After the young man followed this, the story was no longer rigid and had fewer odd sentences. He did not put every word on stage, nor did he always take only the top ten; instead he first circled a cluster of the most promising words, then chose the next sentence from within it. The audience found it fresh yet could still follow along.

He also learned to narrow the basket during tense plots and widen it during relaxed journeys. The scope changed with the scene, so the choices had both order and surprise; the audience could not hear the rules, but could feel that the rhythm had steadied. He also learned to narrow the basket during tense plots and widen it during relaxed journeys. The scope changed with the scene, so the choices had both order and surprise; the audience could not hear the rules, but could feel that the rhythm had steadied. He also learned to narrow the basket during tense plots and widen it during relaxed journeys. The scope changed with the scene, so the choices had both order and surprise; the audience could not hear the rules, but could feel that the rhythm had steadied.

The beauty of this word-sampling method is that it neither rigidly sticks to a few candidates nor lets the whole field rush in at random. First circle the most likely cluster, then keep a little variation within it.

Reveal

This story is about: Top-p sampling

Top-p sampling, also called nucleus sampling, selects the smallest candidate set whose cumulative probability reaches p, then samples from it. It adapts better than top-k to different distribution shapes: deterministic tasks have few candidates, open-ended tasks have many. Enterprise agents often use it to control diversity, but it does not solve factuality—it needs to be paired with grounding, citations, tool results and safety filtering.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Cumulative confidence reaches ninety percent: top-p threshold
  • Box size varies with the task: the candidate set adapts based on the probability distribution
  • Select from the box by weight: sample within the nucleus
  • Contraindication table and evidence checks: factual and safety constraints beyond top-p
  • Do not fetishize a fixed number: the advantage of top-p over top-k

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

samplingnucleus samplingtop-ktemperature

Related distinctions

This concept is easily confused with adjacent concepts; it is recommended to continue to the comparison page.

京ICP备2026014272号