Efficient Federated Learning With Mean Block Difference-Based Global Aggregation and Patience-Based Local Training

As neural network models grow larger and more complex, federated learning (FL) faces challenges in terms of communication and computation efficiency. To address these issues, layer-wise learning has been proposed. Existing approaches did not leverage useful properties of layer-wise learning including update locking and variations in convergence rates, thereby resulting in sub-par model performance. To bridge this gap, we propose theFederatedMeanBlockDifference-based global model aggregation approach withPatience-based local training (FedMBDP). We automatically partition the neural network model into uncoupled blocks and progressively train them. Determining which blocks to train and aggregate becomes a critical task. To improve computation efficiency, we propose a patience-based local training algorithm to adaptively select training blocks, reducing computation latency. To improve communication efficiency, we introduce a mean block difference-based global model aggregation algorithm to dynamically select blocks for aggregation to minimize communication latency. We provide the convergence analysis of FedMBDP. Extensive experiments on three widely adopted benchmark datasets show that FedMBDP achieves superior performance compared to six state-of-the-art approaches. It reduces FL training latency by 26.37% compared to the best baseline, while achieving similar test accuracy.