← Back to Concept Contrast

Concept Contrast Fable

Post-training quantization and quantization-aware training: one is losing weight after the fact, the other is controlling weight from childhood

Conclusion first:Everyone wants to quantize when deploying a model, but when you quantize determines how much precision you lose. Post-training quantization compresses the precision of weights after training is complete; it is cheap but may lose accuracy. Quantization-aware training simulates low-precision computation during training, so the model adapts to quantization noise while training; it preserves accuracy better but has a higher barrier. Choose the wrong method and you either burn extra GPU or miss your metrics after launch.

Everyone wants to quantize when deploying a model, but when you quantize determines how much precision you lose. Post-training quantization compresses the precision of weights after training is complete; it is cheap but may lose accuracy. Quantization-aware training simulates low-precision computation during training, so the model adapts to quantization noise while training; it preserves accuracy better but has a higher barrier. Choose the wrong method and you either burn extra GPU or miss your metrics after launch.

Fable

Two ways of altering clothes in a tailor shop

An old-town tailor shop took on a batch of uniform orders. The client required each finished garment to stay within a standard weight, because logistics charges by weight. The shop had two approaches. The first was the old tailor's way: make the garment with regular fabric first, then weigh each piece, and use scissors to trim excess lining and decorative edges until it met the standard. It was fast and worked with any fabric, but after trimming, some parts of the garment would be slightly deformed.

The second approach was brought back by a young master from out of town: use lighter lining and fabric at the cutting stage, repeatedly try on and weigh during sewing, and adjust stitch density and interlining thickness at each step according to the target weight. This method is time-consuming and requires many more samples up front, but the finished garment needs no further cutting and the pattern stays intact.

The old tailor's first batch was delivered first. The customer's spot check found that all weights met the standard, but several garments had too much shoulder lining cut away, so the collars were not stiff enough when worn. The customer did not return the goods, but added a line to the second contract: "While meeting the weight standard, the fit score must not fall below the previous season's standard." That sentence pushed the old tailor's scissors-based approach into a corner.

The young craftsman's second method took over. At the pattern-making stage, he embedded the target weight into every step—he did not wait until the end to check whether it was overweight, but tightened each stitch in advance toward the final weight. When the second batch was delivered, weight and fit met the standard at the same time, and the client transferred all subsequent orders to him.

Someone asked the young master: you spend so much extra time up front making samples and testing patterns, isn't that only suited to large orders? The young master did the math: if the order is too small to be worth re-drafting a pattern, the old tailor's after-the-fact cutting really is more cost-effective. But if the client demands a fit score every time, the cost of redoing samples is far lower than later rework. The key is whether the client wants "weight up to standard" or "both weight and fit up to standard."

The tailor shop later wrote both methods into its order-taking manual. For rush orders, short orders, and orders with low pattern requirements, it could make the garment first and then trim lightly; for long-term customers, repeated acceptance, and orders with pattern scores written into the contract, it had to design to the target weight from pattern-making onward. The masters no longer argued over which scissors were more advanced; they first looked at what consequences the order bore.

At the end of the manual there was one more reminder: post-hoc trimming looks cheap, provided that trimming a little wrong is acceptable; controlling weight in advance looks troublesome, but buys less rework at delivery. If a small order that only needs fast delivery is repeatedly pattern-made, the labor cost will be drained; if a high-requirement large order is left to scissors and luck, sooner or later the whole season's business will be lost in a customer spot check.

Post-training quantization

Compressing the precision of weights and activations after model training is complete, without retraining or modifying the training process; simple to implement but may introduce precision loss.

Quantization-aware training

Simulate quantization operations during training so the model adapts to low-precision representations during the training stage. Accuracy is better preserved, but additional training compute resources are required.

Story Mapping

  • Make the garment first, then trim the excess weight: PTQ, compressing precision after training is complete
  • At the tailoring stage, repeatedly test patterns with light material: QAT, simulating quantization noise during training
  • The Old Tailor's Shoulder Lining Was Over-Trimmed: PTQ can lose precision on unimportant structures
  • The young master spends more time early but the pattern remains intact: QAT requires more upfront investment but delivers more stable quality
  • Small scale use PTQ, large scale with format requirements use QAT: the scenario determines the strategy choice

Applied to model deployment plans

Before making a quantization plan, first confirm two conditions: whether the model needs a long-term stable service precision commitment after launch, and whether there is budget to run an additional round of training. If you only need rapid validation or a demo, PTQ is enough. If precision metrics are signed in the contract and the customer will conduct regular evaluations, go with QAT. Writing these two items into the first line of the deployment Checklist can prevent precision regressions from disrupting the schedule after launch.

京ICP备2026014272号