Concept Contrast Fable

Quantization and post-training quantization

Conclusion first:Quantization is a general technique for converting model parameters from high precision to low precision to reduce computation and storage. Post-training quantization is performing the quantization conversion after model training is complete—it is only one timing choice for quantization, not the whole of quantization. Quantization can also be done during training (QAT), or weights can be constrained before training. Assuming that “quantization” means PTQ by default will, because other options are missed, make it impossible to achieve the optimal trade-off between latency and accuracy.

Quantization is a general technique for converting model parameters from high precision to low precision to reduce computation and storage. Post-training quantization is performing the quantization conversion after model training is complete—it is only one timing choice for quantization, not the whole of quantization. Quantization can also be done during training (QAT), or weights can be constrained before training. Assuming that “quantization” means PTQ by default will, because other options are missed, make it impossible to achieve the optimal trade-off between latency and accuracy.

Fable

The plate presser and the photographer

Master Qi at the printing house had a special skill: he could compress a catalog image from two hundred megabytes down to twenty megabytes with no visible difference in print quality. After the new typesetter, Xiao Ding, learned this process, he handed all images that needed compression to Master Qi after typesetting was finished, believing this was the only correct way to do it.

Once the print shop received a photography annual: three hundred pages in full color, each photo a forty-megabyte RAW file. Xiao Ding finished the layout using the old process and sent it to Master Qi for image compression. Master Qi compressed it for two days, but the resulting file was still too large and the printing press could not run it. Xiao Ding asked whether it could be compressed more aggressively. Master Qi tried, but the color shift was severe, and the photographer refused.

Old Gu, the print shop's technical director, looked at the process and saw the problem wasn't Master Qi's compression technique—it was "when to compress." RAW files hold a great deal of raw data: shoot a low-light scene and the RAW stores the complete sensor readings, with rich detail in the shadows. If layout ignores all this and takes the RAW straight to typesetting, compressing only at the end, it's like driving a fully loaded truck into an alley before unloading—you can't get in and you can't finish unloading.

Old Gu changed the process. Before photos enter the layout system, they go through preprocessing: reduce resolution to the precision required for printing, crop out the area beyond the bleed line, and convert the color space from sixteen channels to four. After preprocessing, the file had already gone from forty megabytes to eight megabytes. Master Qi compressed it another round, down to two megabytes, and the print quality actually improved—because preprocessing removed only the information that was not needed for printing.

Only then did Xiao Ding understand: compression can be done at different stages, and each stage involves different trade-offs. Compressing after typesetting is like "cleaning up afterward"—the advantage is simplicity, the disadvantage is that every earlier step carries precision it does not need. Compressing at the preprocessing stage is like "slimming down in advance"—the advantage is that all later steps become lighter, the disadvantage is that you have to know which information will truly not be needed later.

Old Gu later introduced a third approach: for certain illustrated albums with high reprint rates, shoot directly at a lower bit depth during the photography stage, controlling the data volume at the source. Xiao Ding finally saw the whole picture - compression can happen at different points along an entire pipeline, and different intervention points lead to different consequences.

Xiao Ding drew a diagram in his notebook: the horizontal axis was the different stages of the pipeline, and the vertical axis was compression strength. He told the new colleague: you used to think compression only happened after typesetting was finished, but in fact it can be done during shooting, before entering the system, before typesetting, or after typesetting. The cost and benefit of each intervention point are different, and you have to choose based on the specific project.

Old Gu added one last point: the image-compression master's skill was fine—the problem was tying that skill to a single point in time. Ask him to unload after the truck is already stuck in the alley, and even a miracle worker can't unload cleanly. Shift the timing so he handles it in advance, and the same skill produces a completely different result.

Quantization

A general technique for converting model parameters and computation from high-precision data formats to low-precision formats, with the goal of reducing computation and storage requirements while keeping accuracy loss acceptable. It is an umbrella term for a class of methods that includes various intervention timings and implementation strategies.

Post-training quantization

A specific method for quantizing and converting already-trained weights after model training is complete. It is simple to implement and requires no retraining, but may cause significant precision loss. It applies only to certain model architectures and scenarios where precision requirements are not extreme.

Story Mapping

Xiaoding assumes all compression happens after layout: treating PTQ as the only form of quantization.
Formatting RAW files directly puts the whole pipeline under load: the limits of PTQ — every earlier stage runs at full precision.
In the preprocessing stage, reduce precision first, then lay out: quantization-aware training or other pre-quantization strategies.
Shoot at a lower bit depth during the filming stage: quantization constraints before training, controlling from the source.
Multiple intervention points on the horizontal axis: quantification is not a single point in time, but an entire chain of optional strategies.

In the project

When deciding to do quantization, first ask about the timing of intervention. PTQ is the fastest option to get started, but if you're willing to invest training resources, QAT usually preserves more precision; if your model architecture itself supports low-precision training, you can also constrain it directly during the training phase. Don't equate "quantization" with "PTQ" at the solution evaluation stage—otherwise you'll miss options that require more engineering investment but offer greater returns.

Quantization Post-training quantization