On-Device Federated Learning via Second-Order Optimization with Over-the-Air Computation

Federated learning becomes a promising approach for preserving privacy by keeping user data locally. The basic idea is that a central server iteratively aggregates distributed local models trained directly on mobile users' local datasets to form a high-quality global model by computing the weighted sum of the locally updated models. However, the communication cost becomes the main bottleneck as a large number of communication rounds are involved in the federated learning procedure. We propose to update local models by second- order optimization methods with fast convergence rates, thereby significantly reducing the communication rounds for global model updates. Furthermore, the over-the-air computation technique is adopted to improve communication efficiency for model aggregation by utilizing the superposition property of wireless channels. A nonconvex low-rank beamforming approach is then developed to support over- the-air computation via difference-of-convex-functions (DC) programming. Through extensive experiments, we reveal that the proposed DC algorithm is able to significantly minimize the aggregation error, and the second- order methods are quite robust to the model aggregation errors.