SOURCE-LINKED INTELLIGENCE

Grouped Value Attention: Efficient KV Caching via On-Demand Key Reconstruction

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

The KV cache is a primary bottleneck for Transformer decoding: its memory footprint and cache-read traffic grow with sequence length. Grouped-query attention (GQA) reduces this cost by sharing key-value heads, but still stores both a key and a value at every step. We introduce Grouped Value Attention (GVA), which stores grouped values and reconstructs content keys with a learned linear map. At inference, the map can be absorbed into the query, eliminating the need to materialize content keys in the intended decode path. A small shared decoupled RoPE channel retains positional information throu

Read original source ↗ Open in workspace

recordType

paper

region

Global