SOURCE-LINKED INTELLIGENCE

Structure is not mechanism: high-gain gated-FFN rows across text and genomic foundation models

arXiv · AI, language, vision and robotics · article · Sep 13, 2026 · UTC

A small number of unusually high-gain parameters can exert disproportionate effects in transformer language models, but whether analogous structures recur in genomic foundation models and whether structural geometry determines functional importance remains unknown. We analyzed high-gain rows in gated feed-forward networks across text and genomic foundation models, including a frozen 22-model causal census. Computing an associated bilinear weight operator exactly, without a diagonal approximation, we tested whether structural extremeness is a transferable mechanism. Activation-derived candidate

Read original source ↗ Open in workspace

recordType

paper

region

Global