FedGPD: Global Prototype Distillation InHeterogeneous Federated Learning

Abstract Federated learning is a distributed machine learning paradigm where the goal is to collaboratively train a high quality globalmodel while private training data remains local over distributed clients. However, heterogenous data distribution over clientsis severely challenging for federated learning system, which severely damage the quality of model. In order to address thischallenge, global prototype distillation (FedGPD) for heterogenous federated learning is proposed to improve performance ofglobal model. The intuition is to use global class prototypes as knowledge to instruct local training on client side. Eventually,local objectives will be consistent with the global optima so that FedGPD learns an improved global model. Experiments showthat FedGPD outperforms previous state-of-art methods at the cost of a slight communication overhead.