SOURCE-LINKED INTELLIGENCE

What Does Layer-Importance Reveal About Transformers and State-Space Models?

arXiv · AI, language, vision and robotics · article · Sep 15, 2026 · UTC

Transformers and state-space models (SSMs) are the two dominant families of sequence models, and a central open question is how far the analytical knowledge built for transformers transfers to SSMs. We address this through the lens of layer importance which underpins compression, selective fine-tuning, and interpretability across both families. We decompose layer importance into two distinct notions. \emph{Necessity} captures how much the pretrained model depends on a layer's existing contribution, measured by the loss increase from bypassing it. \emph{Plasticity} captures where the model abso

Read original source ↗ Open in workspace

recordType

paper

region

Global