SOURCE-LINKED INTELLIGENCE

Cache-Aware Joint Router Adaptation for Memory-Efficient MoE Inference

arXiv · AI, language, vision and robotics · article · Sep 4, 2026 · UTC

Mixture-of-Experts (MoE) models activate few experts per token, yet their full expert sets can exceed GPU memory and require repeated weight transfers during decoding. We formulate expert-cache management as a model-side algorithmic problem and propose cache-aware post-training that jointly adapts the MoE backbone and lightweight auxiliary routers while preserving the native inference-time Top-K rule. The update-only Temporal Router learns same-layer retention across tokens without proactive loading. The full Spatio-Temporal Router adds a Spatio Router that uses the causal predecessor's hidden

Read original source ↗ Open in workspace

recordType

paper

region

Global