Context-Aware Online Model Splitting and Device Association for Semi-Decentralized Federated Learning in Internet of Things

As a distributed approach to Artificial Intelligence (AI) model construction over wireless networks, federated learning (FL) based on multi-device collaborative training can protect data privacy, as well as increase the computing load of local model updates. In contrast, split learning (SL) with proper model splitting can adapt to the computation and transmission capabilities among devices. In this paper, while taking advantage of FL and SL, we concentrate on a semi-decentralized hybrid federated split learning (SD-HFSL) framework, in which we surpass the limitations of a single central server and allow the shared split models to be aggregated among multiple edge servers. To verify the importance of latency optimization for training efficiency, we analyze the convergence performance of SD-HFSL while jointly considering the limited computation and communication resources. Then, aiming at maximizing the long-term training efficiency, we propose an online optimization problem that includes local model splitting and device association. Considering that the training latency is unknown to the system a priori, a context-aware online training algorithm with sublinear regret is proposed based on the framework of contextual multi-armed bandit (CMAB), where the edge servers can observe the context information of device sites for latency estimation, followed by the iterative optimization based on the evaluated information in different contexts. Experiments on several neural network models show that the proposed algorithm reduces training latency and improves test accuracy compared with the selected benchmarks.