Answers are grounded in this paper and cite their sources.
Federated learning is a distributed machine learning paradigm designed to facilitate collaborative model training while preserving user data privacy. However, in practical scenarios, data and model heterogeneity have emerged as significant barriers to its widespread application. Existing model heterogeneous personalized federated learning (MHPFL) algorithms often face trade-offs between performance and cost of computing and communication. To address this, we propose FedMoEKD, a novel MHPFL algorithm based on the Mixture of Experts (MoE) framework with a dynamic routing mechanism and decoupled training strategy. This approach decouples local MoE training and enhances the specialization capabilities of local expert models through knowledge distillation. By focusing on expert specialization, it achieves superior performance compared to state-of-the-art methods while maintaining minimal computational and communication overheads, including at least 0.58% higher accuracy on CIFAR-10 and 58% lower computation overheads than FedProto. Extensive experiments conducted on multiple datasets demonstrate that FedMoEKD outperforms existing MHPFL approaches in both convergence speed and model accuracy.
No takes yet. Share an insight, caveat, or question.
Liang et al. (2025) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: