SOURCE-LINKED INTELLIGENCE

Sparse Competition during Training For the Emergence of Specialized Modules

arXiv · AI, language, vision and robotics · article · Aug 31, 2026 · UTC

Modularity in deep neural networks has been proposed as a means of improving both interpretability and training by promoting disentangled representations and reducing redundancy. In this work, we study the emergence of modular structure through competition dynamics between groups of neurons during training. We introduce a method that (i) maintains near-baseline accuracy, (ii) induces usage-based modularity by sparsely routing inputs to neuron groups, and (iii) encourages specialization of these modules, such that their activations are correlated with input classes. We evaluate the proposed app

Read original source ↗ Open in workspace

recordType

paper

region

Global