← Back to concept explainers

Concept Fable

NF4

Conclusion first:NormalFloat 4-bit (NF4) is a 4-bit data type introduced with QLoRA, designed for normally distributed weights. Instead of using uniformly spaced values, NF4 allocates representation more effectively for typical neural weight distributions. It helps load and fine-tune large models in 4-bit form with less quality loss.

NormalFloat 4-bit · Quantization

Read the story first. There is no rush to give a definition here; let the question grow on its own first.

Fable

The silver vault made four-digit scales for common weights

The royal treasury kept many small silver ingots. The old scale had very fine markings, so any weight could be recorded precisely, and the accountants used it with confidence.

Later, the number of silver ingots exploded, and the ledger could no longer hold them. The chief steward decided to shorten the weight records, keeping only four digits. The chief steward thought that a shorter ledger meant success, until a stocktake revealed that mid-range weights were the easiest to confuse, and these made up the majority of the vault.

The apprentice first divided the scale evenly into sixteen slots. He soon found that most silver ingots clustered around the middle weights, and the even scale wasted precious slots on the very light and very heavy that rarely appeared.

Someone proposed recording only whole numbers. The ledger became shorter, but the subtle differences between common silver ingots were erased, and inventory discrepancies kept accumulating. The whole-number method looked more economical, but disputes arose at payout. Two silver ingots that were similar but of different purity were recorded as the same grade, and the merchants refused to accept it.

The old accountant examined years of weight distribution and found that silver ingots clustered in the center like a bell curve. He placed more of the sixteen marks in the common range and left a few marks for edge weights.

The new scale was not uniform, yet it fit the real silver ingots in the vault better. The most common weights could be subdivided, and rare weights also had a place. Before the new scale went live, the old accountant repeatedly simulated with old ledgers, comparing which ingots would be recorded incorrectly, by how much, and where. Only after confirming the loss in common ranges was minimal did they switch the whole vault's ledgers.

After the ledger was shortened, inventory counting remained stable. The chief steward finally understood that for low-digit records to lose less, the scale must be close to the data's own distribution. The apprentice later understood that four-digit scales aren't magic. They save ledger space, but also lose some precision; the cleverness lies in arranging the loss where it matters less.

The apprentice also tried making the middle scale denser and leaving only a little room at the edges. Counting common silver ingots became more accurate, but extreme weights were occasionally recorded incorrectly. The old accountant had them test repeatedly with the real silver vault, rather than being confident based on drawn scales. The vault did not invent more digits; it simply used the four digits in the most valuable positions.

Reveal

This story is about: NF4

NormalFloat 4-bit (NF4) is a 4-bit data type introduced with QLoRA, designed for normally distributed weights. Instead of using uniformly spaced values, NF4 allocates representation more effectively for typical neural weight distributions. It helps load and fine-tune large models in 4-bit form with less quality loss.

What matters is that it is not just a term, but a structural problem that repeatedly appears in real AI / Agent systems. Understanding it lets you judge when to add a model, when to change a process, and when to add governance.

Metaphor mapping

  • Silver ingot weight: model weight values
  • Four scales: 4-bit representation
  • Average sixteen grids: ordinary uniform quantization
  • Bell-shaped distribution: the approximate normal distribution common in neural network weights
  • Denser scale near the center: NF4 quantile/non-uniform representation
  • The ledger stays stable after shrinking: preserving quality when loading with low VRAM or fine-tuning with QLoRA

Andy Ren Xu's judgment

The practical value of this concept is that it helps you break "AI features that look smart" into deliverable, verifiable, and governable units of work.

QLoRAbitsandbytesquantization
京ICP备2026014272号