← Back to concept explainers

Concept Fable

Logits

Conclusion first:Logits are the raw scores a model outputs before softmax converts them into probabilities. Decoding parameters, constraints, banned words, temperature scaling, and sampling strategies all act on logits or the distributions converted from them. Understanding logits helps explain why a model favors certain tokens, and also supports sensitive word suppression, format constraints, confidence analysis, and generation debugging in enterprise scenarios.

Logits · Decoding

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

Before the bronze scale was replaced with a price tag

The herbal medicine shop had a candidate tray. Whenever the apprentice needed to make a prescription, the old master first placed the possible herbs on the tray and put a small stone of varying height next to each herb. The higher the stone, the more it leaned toward being chosen at that moment. The stones on the tray changed every day. As customers' symptoms differed and seasons changed, the heights beside the herbs also differed; if the apprentice merely memorized yesterday's highest, he might pick wrong today.

When the new apprentice saw the stones, he assumed the highest one would surely go into the formula. The master shook his head: the stones were not yet the final drawn shares, only raw tendencies. Some medicines differed greatly, and once converted to lots they were almost certain to be chosen; others differed only slightly, and after conversion still had multiple possibilities.

To be safe, the shopkeeper once required taking only the highest-flavor medicine each time. For ordinary minor illnesses it was fine, but for complex symptoms it seemed rigid. Later he had the apprentice randomly draw from all medicines, and as a result a very low-grade small stone was drawn into the prescription, nearly causing an error. The random draw was especially frightening. A potent medicine that should almost never enter a prescription was drawn, and the master stopped it in time, so the customer did not suffer.

The old master taught them to first replace the small stones with drawable shares, and then add rules by scenario: emergencies could lean toward the highest, and when diverse prescriptions needed to be tried, the second-highest medicine could also have a chance; if only the first few candidates were retained, medicines outside the plate could no longer enter. Changing to shares would also change the temperament. If the shares were too sharp, the highest would be chosen almost every time; if too flat, low-preference medicines would also often pop up.

In a retrospective, a guest asked why a certain herb had not been chosen. The master did not look only at the final draw, but pulled out the height of the small stones at the time: that herb had originally leaned very low, and it was the later drawing rules that knocked it out completely.

When the shopkeeper later asked why the formula had changed, the apprentice would bring out two things at once: the height of the small stone at the time, and how it was later changed to a lottery share. Looking only at which herb was finally obtained, one cannot see the tendency before the choice.

The apprentice finally understood that the initial high or low on the candidate board is not the answer, but it determines where the answer comes from. To explain a choice, one must look both at the original tendency and at how it was converted, filtered out, or drawn.

Reveal

This story is about: Logits

Logits are the raw scores a model outputs before softmax converts them into probabilities. Decoding parameters, constraints, banned words, temperature scaling, and sampling strategies all act on logits or the distributions converted from them. Understanding logits helps explain why a model favors certain tokens, and also supports sensitive word suppression, format constraints, confidence analysis, and generation debugging in enterprise scenarios.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Raw weight difference of the bronze scale: the model output logits
  • Prescription rules and contraindications: business constraints and safety filtering
  • Switch to probability: softmax and other distribution transformations
  • Mistaking raw scores for the final choice: confusing logits with output probabilities/answers
  • Analyzing word-frequency anomalies: analyzing generation behavior through logits

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

softmaxtoken probabilitydecodingtemperature
京ICP备2026014272号