SOURCE-LINKED INTELLIGENCE

MoRE: Mixture of Reused Experts

arXiv · AI, language, vision and robotics · article · Sep 16, 2026 · UTC

Mixture-of-Experts (MoE) architectures decouple model capacity from computational cost, yet incur high memory footprints as parameters grow linearly with the number of experts. Recurrent Transformers achieve parameter efficiency by reusing layer weights, but typically lack the capacity for competitive language modeling. We propose Mixture of Reused Experts (MoRE), a hybrid that shares expert pools across groups of adjacent layers. Each layer retains its own router but selects from a larger shared pool, expanding the diversity of routing combinations without additional parameters. To enable sha

Read original source ↗ Open in workspace

recordType

paper

region

Global