FedHDE: an efficient federated learning model based on hyper-knowledge distillation

The heterogeneity of data between different clients limits the performance of federated learning training, and hyper-knowledge distillation has emerged as a potential solution to address heterogeneous data. However, existing hyper-knowledge distillation methods often sacrifice efficiency to improve model performance. To address these challenges, this paper proposes FedHDE, a novel federated learning framework based on hyper-knowledge distillation and efficient tensor optimization. FedHDE integrates a Dynamic Client Selection strategy within a teacher–assistant–student hierarchical structure to enhance global aggregation accuracy and reliability without relying on public datasets or server-side models. Furthermore, an efficient tensor optimization algorithm is introduced to improve numerical stability and reduce computation time. Theoretical analysis demonstrates that FedHDE achieves a convergence rate of O(1/√T) O ( 1 / T ) , ensuring efficient and stable learning. Experimental results on the CIFAR10 dataset show that FedHDE improves both local and global accuracy by 0.14%–4.32%, while reducing training time by 65.28% compared to state-of-the-art baselines.Code is public at https://anonymous.4open.science/r/FedHDE .