SOURCE-LINKED INTELLIGENCE

LoGo: Token-Level Dynamic Local-Global Attention

arXiv · AI, language, vision and robotics · article · Aug 30, 2026 · UTC

As context lengths scale, attention increasingly becomes a primary computational bottleneck in large language models. Standard Transformers remain powerful but computationally inefficient, as they allocate the same attention budget to every token regardless of its contextual demand. Existing local-global hybrids provide a more efficient alternative by mixing restricted- and full-context attention, but they typically allocate span statically across layers or heads. To address these limitations, we propose LoGo, a token-level dynamic local-global attention mechanism that uses attention span as a

Read original source ↗ Open in workspace

recordType

paper

region

Global