SOURCE-LINKED INTELLIGENCE

Quality-Constrained Routing over a Fixed Pool of Quantized Mixture-of-Experts Instances

arXiv · AI, language, vision and robotics · article · Sep 11, 2026 · UTC

Quantized Mixture-of-Experts (MoE) services can hold several pre-materialized instances of one base model, but quantization damage varies sharply across requests and bitwidths. Because instance materialization and replica counts consume memory and require slow reconfiguration, we treat them as upstream provisioning decisions and study routing within a fixed resident pool. Within this fixed-pool boundary, we route each request to maximize modeled throughput under a class-level expected quality-degradation budget and measured instance capacities. To predict this request-specific risk, we introdu

Read original source ↗ Open in workspace

recordType

paper

region

Global