When Does Trace-Driven Evaluation Mislead MoE Expert Caching? Replay Semantics, Workload Contamination, and Operating Regimes

Mixture-of-Experts (MoE) models have outgrown the high-bandwidth memory of the accelerators that serve them, and offloading expert weights to host memory has become a standard response. This makes expert cache management an attractive lever: if a smarter policy raised the hit rate, the same model would need lessexpert traffic per token. Evaluating that hypothesis is a measurement problem, and we find the measurement fragile. Using a trace-driven, event-atomic simulator over three MoE models (40, 64 and 128 experts), we isolate three evaluation axes that change conclusions ratherthan shift numbers. Replay semantics: under a fused-event traffic contract, an inconsistent per-access replay inflates recency-based policies by 27-29% whileleaving frequency-based and static policies within 4%, inverting the policy ranking. Workload contamination: probe sets using one instruction template percategory produce verbatim-identical generation prefixes; a matched-pair rendering intervention moves the measured early-window effect by 19.4-31.9percentage points and reverses which workloads appear most cache-friendly.Operating regimes: normalized miss fractions do not transfer across models, so the per-step expert union relative to per-layer capacity must be reported,though permuting only the temporal order of an otherwise identical event stream moves the offline-optimal gap from 44.9% to 30.8%, so it is not sufficient. After correcting all three, a stable gap to the offline optimum remains (44.2-45.9% across 13 frozen workload compositions). A forced-admission oracleattributes 84.3-96.6% of it to knowing which resident expert is used furthest in the future. A causal next-use predictor, used as an eviction rule, recovers-11.4% of the gap; at the decision points it selects an optimal victim 3.4% of the time against 2.4% for a random resident block and 20.6-22.1% for LRU andLFRU. Our position is narrow: in our evaluated settings a large offline-optimal gap substantially overstates the gains recovered by representative lightweightcausal mechanisms. All results are trace-driven simulation. No real host-to-device transfer, interconnect topology, kernel time or latency was measured, and no deploymentclaim follows from any result. This deposit records the manuscript together with SHA-256 hashes of the simulator, the diversity-controlled workload probeset, the contamination diagnostics and the frozen result manifests, so that a later public release can be verified against this record.