A more recent development within machine learning is a technique called federated learning. Instead of training a model on a central server, a model is instead collaboratively trained on several devices. The data on each device is not shared with the server but instead kept locally on the devices, which increases data privacy and security for the users. Federated learning was introduced with the optimisation algorithm Federated Averaging (FedAvg), that uses a local stochastic gradient descent before communicating the updated weights to the server. However, FedAvg struggles in setups where the data distribution is unbalanced, and therefore other optimisation algorithms have been developed to address these issues. In this project, we aim to test different optimisation algorithms and study their accuracy and loss on different datasets. Specifically, the focus has been on heterogeneous data and fairness of the model performance. The optimisers were analysed and compared on different levels of data heterogeneity. Our analysis identifies the optimisation algorithm best suited for certain setups and concludes the overall best performing algorithm to be FedAvg-M.
Kristiansson et al. (Wed,) studied this question.