How Faithfully Does Routing Imbalance Predict the Cost of Fixed-Capacity Dispatch in Mixture-of-Experts Models? An Empirical Characterization

Mixture-of-experts (MoE) layers route each token to a small subset of expert sub-networks. A common implementation, fixed-capacity dispatch (as in DeepSpeed), pads every expert's slice of a dense buffer to a common size and drops tokens that overflow it. Under a zero-drop capacity construction (buffer size equal to the busiest expert's load), the ratio of padded work to real work is exactly P = max(tokens per expert) / mean(tokens per expert), an algebraic identity rather than an empirical discovery. This paper asks how faithfully realized GPU cost tracks that ratio, and where the tracking breaks down. It proposes no new execution mechanism; systems such as MegaBlocks already avoid padding. Within a controlled regime, cost tracks P closely. Under synthetic routing that sets P directly, fixed-capacity latency grows with P while padding-free latency shows at most weak dependence. On real Triton grouped-GEMM kernels, kernel-phase latency has a log-log elasticity of about 1.05 with respect to P (a point estimate without a valid interval), close to the idealized value of 1. The mapping to realized speedup is not architecture-independent. On real routing traces from three model families (OLMoE-1B-7B, Qwen1.5-MoE-A2.7B, Mixtral-8x7B) replayed through a Python-loop reference (8,832 nested observations), leave-one-model-out prediction fails: out-of-sample R² is negative for every family. In a smaller real-kernel run (83 layer-level observations, four families including Switch-Base-32), held-out sign fails for Switch-Base-32 and Mixtral, and degenerate Switch decoder points may drive part of this. In the synthetic sweep, expert count adds substantial variance beyond P at high imbalance, while top-k changed speedup by at most 6% across completed cells and no effect of load shape was detected under a descriptive criterion. Fixed-capacity dispatch uses 1.6x-6.2x the memory of padding-free dispatch. Cross-model claims rest on three or four families, not thousands of rows. Several limitations remain open and are documented in the paper, including unresolved correctness-check anomalies, an unrecorded software environment, and no measurement of the model-quality cost of token dropping.