SOURCE-LINKED INTELLIGENCE

QTEA: Ternary LLMs with Sparse Residual Salient Weight and By-Column Optimization

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Weight-only post-training quantization (PTQ) can alleviate the computational burden of serving large language models (LLMs) at scale. However, existing PTQ methods often fail to generalize across models and suffer severe accuracy loss below 2 bits. Many leverage unstructured sparsity to mitigate this loss, but at the cost of regularity and GPU-friendly execution. We present QTEA, a sub-2-bit PTQ framework that quantizes weights into ternary values and uses salient weights as residual error compensators. To maintain hardware efficiency, residuals are assigned to selected columns with semi-struc

Read original source ↗ Open in workspace

recordType

paper

region

Global