SOURCE-LINKED INTELLIGENCE

Kalman Delta Networks: Uncertainty-aware Associative Memory

arXiv · AI, language, vision and robotics · article · Sep 7, 2026 · UTC

Linear attention is increasingly used in frontier language models for efficient long-context inference and constant-memory decoding. Its fixed-size recurrent memory, however, requires an online decision at each token: what to write and how strongly to overwrite existing associations before knowing which information future queries will require. Delta-rule models learn this strength from the current token embedding but do not track confidence in the memory estimate, preventing each write from adapting to accumulated evidence. To represent this uncertainty explicitly, we reformulate recurrent ass

Read original source ↗ Open in workspace

recordType

paper

region

Global