SOURCE-LINKED INTELLIGENCE

HyQuant: Hybrid-Precision Quantization for LLM Attention

arXiv · AI, language, vision and robotics · article · Aug 28, 2026 · UTC

Quantization has been widely adopted in LLM training and inference to reduce cost and improve efficiency. However, low-bit quantization of the \emph{attention} module often introduces large errors at very low bit-widths, causing performance degradation. Existing methods mainly rely on smoothing techniques to handle outliers, while we propose a hybrid quantization design to better balance accuracy and efficiency. Specifically, we propose \textbf{HyQuant}, an efficient hybrid quantization framework for LLM attention. HyQuant quantizes most attention states into low-bit formats while retaining a

Read original source ↗ Open in workspace

recordType

paper

region

Global