TopoMoE: Decentralized Mixture-of-Experts

This revised manuscript proposes an upgraded TopoMoE framework, a decentralized Mixture-of-Experts architecture built on kernelized tensor topology and discrete exterior calculus. Traditional decentralized MoE models adopt fixed tensor fusion and static topological constraint weights, failing to adapt to variable inter-expert coherent coupling strength, which leads to unstable information flow and cyclic logical conflicts under intensive closed-loop expert interactions. To address this limitation, this work introduces the normalized macro effective duality coupling metric Ceff⁽ᵉ⁾ to quantitatively distinguish weak decoherent and strong coherent interaction regimes between experts, enabling adaptive switching of message fusion strategies. Classical Concat+MLP tensor fusion is applied under weak coupling, while the proposed Def-Fuse deformed non-commutative fusion module activates for strong coherent states. Def-Fuse incorporates a learnable twist angle θ̃ to generate asymmetric interaction terms, capturing non-commutative topological correlations between expert tensor states and neighbor messages; the twist angle is only updated in high-coupling scenarios to control training overhead. Furthermore, we design dynamic topology constraint weights governed by Ceff⁽ᵉ⁾: curl and curvature loss weights are amplified in strong coupling regimes to suppress cyclic conflicts, forming a revised total loss function integrating gradient, divergence, curl and curvature regularization. A complete end-to-end training algorithm based on simplicial complex construction is elaborated. The proposed adaptive coupling, Def-Fuse module and dynamic loss weights maintain full backward compatibility with the original TopoMoE—all novel components vanish when coupling strength or twist angle approaches zero, degenerating to the vanilla fixed-parameter framework. All modeling modules are independent macroscopic engineering components without quantum spacetime algebra dependencies, and fully validated on decentralized multi-expert collaborative tasks. This work is first released, archived and updated on Hongqiao University Science and Technology Academic Network. The site provides the official long-term version of this manuscript.