Modular Model Surgery in Sparse Mixture-of-Experts via Surgical Expert Grafting and Contrastive Router Re-anchoring (SEGR)

Sparse Mixture-of-Experts (MoE) architectures achieve unprecedented parameter scaling while maintaining constant FLOP efficiency during inference. However, continually adapting pre-trained MoEs to downstream specialized domains remains fundamentally bottlenecked by Catastrophic Forgetting and the Router Blindness Phenomenon. In this work, we introduce SEGR (Surgical Expert Grafting & Contrastive Router Re-anchoring), a zero-retraining framework for modular MoE model surgery. SEGR operates via four unified mathematical innovations: (1) Target expert excision and activation-manifold adaptation via orthogonal low-rank residual adapters, isolating domain knowledge without perturbing base weights; (2) Contrastive Router Re-anchoring (CRR), a strictly convex contrastive loss equipped with a projected gradient mask, which steers domain tokens to the grafted expert while leaving non-target expert routing boundaries mathematically invariant; (3) Multi-Layer Cascading Re-anchoring (MCR), establishing an unbroken Domain Highway with bounded cumulative routing error; (4) Contextual Polysemantic Routing (CPR), resolving polysemantic bleed and hard-suppressing punctuation leakage to <= 0.02%; and (5) Capacity-Bounded Elastic Fallback (CBEF), eliminating 100% of token drops (0.00% drop rate) under burst overloads. Validated on open frontier foundation MoEs (allenai/OLMoE-1B-7B-0924 and Qwen1.5-MoE) on authentic Bengali corpora, SEGR achieves state-of-the-art domain routing (98.37%) with virtually zero generalist degradation (+0.08% English PPL drift) on an NVIDIA Tesla T4 GPU.