Merging Subgroup Information to Supplement Personal Information for Personal Federated Learning via Model Clustering

Abstract Personalized federated learning represents a pivotal strategy for addressing the challenges posed by statistical heterogeneity in federated learning. Clients optimize their models by leveraging information from other clients through a global model. Despite clients' expectations of acquiring requisite information from global aggregation, this process inevitably leads to a loss of personalized information, particularly impacting clients with limited dataset. Consequently, the acquisition of sufficient personalized information to enhance local models becomes arduous, thereby compromising the efficacy of model personalization. In response to this challenge, we propose the Federal Merging Subgroup Information (FedMSI) method to augment personalized information in personalized federated learning. FedMSI leverages model clustering to delineate subgroup divisions of akin personalized models, aggregates cluster center models based on these divisions and enriches personalized information by incorporating subgroup information from the cluster center models. Experimental findings demonstrate that FedMSI surpasses nine state-of-the-art methods by 7.89% in terms of accuracy under identical data heterogeneity conditions, while exhibiting a 20.34% enhancement when the client data volume is small. Finally, through ablation experiments, this study reaffirms that the augmentation of personalized information through subgroup information significantly enhances the classification performance of personalized models.