Fed-Frenemy: robust screening of compromised clients, balancing heterogeneity

Federated learning has emerged as a new paradigm of distributed learning that trains models jointly without exchanging data. However, it still faces the risk of heterogeneous data distribution and backdoor attacks. Generally speaking, due to the data heterogeneity, the local convergence to the local optimum usually deviates from the global optimum, resulting in the drift of federated learning and hindering the efficiency of model aggregation. Moreover, due to the possibility of different degrees of backdoor attacks, the addition of malicious clients interferes with the global aggregation or even does not converge. We propose FedFre an automatic weight adjustment method to aggregate globally submitted models. This new meta-learning algorithm used on federated learning assigns weights to each client based on its gradient direction. To assign this weight, we perform a meta-gradient descent step on the clean, unbiased few-shot dataset owned by the server to minimize this loss. Applying this technique, we get impressive results both for client aggregation of heterogeneous data distributions and for client aggregation under a large proportion of attacks.