Prototype Similarity Distillation for Communication-Efficient Federated Unsupervised Representation Learning

Federated unsupervised representation learning aims at leveraging unlabeled data from multiple parties to learn visual representations without compromising the data privacy and tackle the non-IID challenge by aligning diverse representation spaces. However, model heterogeneity and communication overhead will directly impact the convergence rate and model accuracy of federated unsupervised learning. And it is challenging to learn visual features for downstream tasks under the premise of compatibility with heterogeneous models and reducing communication overhead. To address these issues, we propose a novel communication-efficient federated unsupervised representation learning framework based on prototype similarity distillation (FLPD). In this framework, the global model builds the feature representation space based on the global dataset and steers the optimization of the prototype relations of the client models. In addition to employing discriminative self-supervised learning for model training, each client fine-tunes the local representation space with global prototype similarity via knowledge distillation, which facilitates local models to fit both the local data distribution and the global representation space. In order to maintain the compactness of prototypes within the same category and enhance the separability between prototypes of different categories, a prototype-based consistency constraint is introduced to alleviate the conflict between local and global representation space. Experimental results demonstrate that our framework outperforms other alternative approaches in terms of communication efficiency and accuracy in the federated settings with statistical heterogeneity and model heterogeneity.