SOURCE-LINKED INTELLIGENCE

Why Does Post-Training Quantization Work?

arXiv · AI, language, vision and robotics · article · Sep 10, 2026 · UTC

Post-training quantization compresses large language models (LLMs) by storing their weights at reduced precision, and each quantized weight introduces an error into the hidden states. Naively, these errors should accumulate with depth and corrupt next-token prediction; randomly initialized models accumulate these discrepancies rapidly, whereas quantized pretrained models accumulate much less hidden-state error and largely maintain downstream task performance, even though they were never trained with quantization noise. This raises the question we address: why does post-training quantization wo

Read original source ↗ Open in workspace

recordType

paper

region

Global