HySync: Hybrid Federated Learning with Effective Synchronization

Federated learning schedules mobile devices to train data locally and upload updates of model parameters to the cloud for aggregation, typically using synchronous algorithms. The synchronous algorithms have been shown to achieve better prediction accuracy as compared to asynchronous algorithms which are more flexible in scheduling but fluctuate greatly in accuracy with the change of staleness. We present HySync, a hybrid algorithm for federated learning to improve the scheduling efficiency and the stability of accuracy. The evaluation result shows that HySync can update the global model flexibly without waiting for each selected client to finish training work. The training time of HySync is 33% shorter than the synchronous algorithm. At the same time, in the case of highly concurrent updates, HySync can control the staleness to half of that of the asynchronous algorithm, leading to 5% higher accuracy than the asynchronous algorithm and the same accuracy as the synchronous algorithm.