SOURCE-LINKED INTELLIGENCE

Nyström Attention Matches Full Attention for Cross-Sectional Stock Prediction

arXiv · AI, language, vision and robotics · article · Sep 8, 2026 · UTC

MASTER's inter-stock multi-head attention -- the module responsible for modeling cross-sectional stock relationships -- accounts for 42.5% of model parameters and 25% of predictive value. We systematically decompose this module and uncover a surprising structure: the learned attention is near-uniform (perplexity 278/300), yet forcing exact uniformity eliminates all cross-sectional discrimination. Spectral analysis resolves this paradox: the deviation from uniformity is low-rank (effective rank ~65, top-10 modes capture 96.5% of energy), explaining why sparse approximations consistently fail wh

Read original source ↗ Open in workspace

recordType

paper

region

Global