SOURCE-LINKED INTELLIGENCE

SeqMoE: Toward Full-Load Performance via Predictive and Graph-Compatible MoE Offloading

arXiv · AI, language, vision and robotics · article · Sep 11, 2026 · UTC

Mixture-of-Experts (MoE) creates a structural advantage for offloading: only a small fraction of activated experts need to reside in device memory, and if they can be loaded in time for computation, offloading can in principle approach full-load performance, where all model weights reside in device memory. Yet translating MoE's structural advantage into practical offloading gains remains challenging. We propose SeqMoE to bridge this gap. To maximize expert hits, we build predictive memory management: (i) Sequence-to-sequence prediction. We are the first to recast expert activation prediction a

Read original source ↗ Open in workspace

recordType

paper

region

Global