SOURCE-LINKED INTELLIGENCE

Revenge of Monosemanticity: Neuron Specialization as a New Form of Feature Learning in MLPs

arXiv · AI, language, vision and robotics · article · Aug 25, 2026 · UTC

Understanding how neural networks learn and organize features is central to understanding their behavior. Much existing theory of feature learning has focused on the emergence of a global low-dimensional representation. We show that this picture is incomplete. In regression problems with clustered data, we demonstrate that multilayer perceptrons (MLPs) naturally develop monosemantic specialized neurons: individual neurons become strongly aligned with a specific predictive feature relevant to a particular region of the input space. Rather than learning a single global low-dimensional representa

Read original source ↗ Open in workspace

recordType

paper

region

Global