← Back to concept explainers

Concept Fable

bf16

Conclusion first:Brain Floating Point 16, bf16 is a 16-bit floating-point format with the same exponent range as fp32 but fewer mantissa bits. Compared with fp32 it saves VRAM and bandwidth, and compared with fp16 it is less prone to numerical overflow, usually without complex gradient scaling. In large-model training and inference, bf16 is often used to strike a balance between stability and efficiency.

Brain Floating Point 16 · Training

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The scale can be coarser, but the range must not be narrow

The herbal medicine shop has two sets of measuring cups. The old measuring cup has very fine graduations, can hold a tiny amount of dew, and can also hold a large jar of thick decoction, but it is very heavy; the new small cup is light and quick, an apprentice can hold two in one hand, but it often overflows when faced with large doses of medicinal liquid.

The proprietor wanted to make dispensing faster, so he first replaced all the counter cups with small ones. On the first day, speed did indeed go up. On the second day, they encountered a large qi-tonifying formula, and the amounts of several medicinal liquids were too large for the small cups to hold, so the apprentice had to convert on the spot, and the more they calculated, the more confused it became.

The old master did not have everyone go back to the old cup. He brought out a third kind of cup: light body, less fine gradations than the old cup, but retaining the old cup's large capacity. It was not good at distinguishing the finest difference, but it was not likely to lose large numbers.

The apprentice worried the prescriptions would become inaccurate as a result. The master took him through them one by one: most tonic soups care about the ratio of medicinal flavors and approximate heat—they don't need every trace to be precise; a few highly toxic, narrow-dose medicines were still left to the old measuring cup.

After a month with the new cup, the counter queue shrank and errors did not increase. The most troublesome case in the past was not a missing grain of powder; when the liquid was too much, the whole prescription would collapse, with heavier consequences. The third cup blocked this kind of collapse.

Later the pharmacy took a large order from the military camp, and several hundred doses of decoction had to be ready before dawn. The apprentice finally saw the use of the third kind of cup: it let the counter move fewer heavy vessels and avoided panic in front of the big pot; only when adjusting the last strong medicine did the old master still personally take out the old cup.

At first, some people remembered only the benefit of "lightness" and wanted to replace all measuring tools. The master craftsman immediately had him prepare an anesthetic, requiring not a half-drop to be off. Only when the apprentice's hand trembled did he realize that coarse graduations were not suitable for every step. What the pharmacy needed was to be fast and stable in most heavy steps, and to preserve the old methods in a few narrow, delicate places. This rule was later posted beside the medicine cabinet, and every time the apprentice took a cup, he first had to check the width of the prescription and the pot volume.

Later the pharmacy set rules: use the light cup for regular formulas, and the old cup for extremely sensitive steps; every cup change must state the reason. The shopkeeper understood that saving effort does not mean casually going coarser — the key is knowing which precision can be given up and which range must never be lost.

Reveal

This story is about: bf16

Brain Floating Point 16, bf16 is a 16-bit floating-point format with the same exponent range as fp32 but fewer mantissa bits. Compared with fp32 it saves VRAM and bandwidth, and compared with fp16 it is less prone to numerical overflow, usually without complex gradient scaling. In large-model training and inference, bf16 is often used to strike a balance between stability and efficiency.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Old measuring cup: fp32, high precision but resource-hungry
  • Ordinary small cup: fp16, resource-saving but with a narrower range
  • The Third Cup: bf16, large range, coarser mantissa
  • Concentrated medicinal liquid overflow: numerical overflow risk
  • Hardware support: the engineering prerequisite for bf16 deployment

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

fp16fp32mixed precisionTPUGPU
京ICP备2026014272号