SOURCE-LINKED INTELLIGENCE

D-Quant: Driftable Entropy Coding for KV Cache Quantization

arXiv · AI, language, vision and robotics · article · Sep 17, 2026 · UTC

The KV cache has become a major bottleneck in deploying LLMs, as its memory footprint grows linearly with sequence length and batch size, imposing substantial pressure on both memory capacity and bandwidth. Among various KV cache compression techniques, quantization is particularly attractive due to its effectiveness and ease of deployment. However, most existing methods rely on fixed-width quantization, where a $b$ bit representation is inherently limited to $2^b$ quantization levels. As the bit width decreases, the number of available levels shrinks exponentially, leading to severe informati

Read original source ↗ Open in workspace

recordType

paper

region

Global