FedCEA: Efficient Adaptive Personalized Federated Learning based on Critical Learning Periods

Abstract Federated learning (FL) faces significant challenges due to statistical heterogeneity, which undermines the global model's generalization capability across diverse clients. Personalized FL (pFL) has been extensively studied to address this, but most existing approaches erroneously assume all stages of the FL training process are equally critical and require the participation of all clients, leading to substantial computational and communication overhead. Addressing this flaw, we propose an efficient adaptive pFL method, FedCEA, based on critical learning periods (CLP). FedCEA tailors efficient models for each client while ensuring data privacy and security by considering CLP during the training process to guide client selection. This approach reduces client-server communications, accelerating model convergence. To enhance the global model's generalization across clients with statistical heterogeneity, we introduce an Adaptive Initialization of Local Models (AILM) module with a personalized aggregation strategy. Additionally, training parameters are dynamically adjusted based on dataset quality to ensure efficiency. FedCEA also employs a compression method that assesses the importance of model parameters, reducing communication and computational costs. To evaluate the effectiveness of FedCEA, we conduct extensive experiments with four benchmark datasets in computer vision and natural language processing domains. The results consistently showed that FedCEA achieves improved accuracy and superior communication efficiency compared to state-of-the-art methods. This makes FedCEA a promising solution for handling heterogeneity in federated learning training. Code is available at https://github.com/buaaYYC/FedCEA/tree/main.