SOURCE-LINKED INTELLIGENCE

OmniKVQuant: KV Cache Quantization for Omni-LLMs

arXiv · AI, language, vision and robotics · article · Sep 10, 2026 · UTC

As Omni-modal large language models (Omni-LLMs) take in audio, video and text together, their KV cache memory cost grows. KV cache quantization is the de facto approach in text-only LLMs, but its application to Omni-LLMs remains unexplored. In this paper, we analyze how TurboQuant, a representative rotation-based KV cache quantization method, behaves on multimodal caches and identify two critical issues: temporal key drift and heterogeneous value geometry. To address these, we propose OmniKVQuant, a training-free framework that (i) sets the key quantization range over each short window of the

Read original source ↗ Open in workspace

recordType

paper

region

Global