Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies

Federated learning is a distributed optimization paradigm that enables a number of resource-limited client nodes to cooperatively train a model data sharing. Several works have analyzed the convergence of federated by accounting of data heterogeneity, communication and computation, and partial client participation. However, they assume unbiased participation, where clients are selected at random or in proportion of data sizes. In this paper, we present the first convergence analysis of optimization for biased client selection strategies, and quantify how selection bias affects convergence speed. We reveal that biasing client towards clients with higher local loss achieves faster error. Using this insight, we propose Power-of-Choice, a communication- computation-efficient client selection framework that can flexibly span the-off between convergence speed and solution bias. Our experiments that Power-of-Choice strategies converge up to 3 \ faster give 10% higher test accuracy than the baseline random selection.