Federated reinforcement learning for scheduling-offloading policies in multi-cluster NOMA systems

Intelligent scheduling and resource allocation of user equipments (UEs) in wireless networks has been an ongoing topic of research. The innovation in this field focuses mostly on generalizing the system to include more components, as well as deriving new ways to solve the problem. We address in this paper an unexplored case of the scheduling-offloading problem for a wireless network with mobile edge computing (MEC). In this network, the UEs have mobility models and are transmitting using non-orthogonal multiple access (NOMA). They are also equipped with data buffers and batteries with energy harvesting (EH) capabilities. We propose a novel UEs clustering approach to account for the growing NOMA inter-user interference, which can lead to performance issues especially in the downlink decoding phase. In addition, clustering can help reduce the problem complexity by distributing it among the clusters that operate independently. We investigate deep reinforcement learning (DRL) to devise efficient policies that minimize the packet loss due to delay infringements. Moreover, we use federated learning (FL) to learn a unified policy accounting for the dynamic nature of clusters. Our simulation results based on DRL method, namely the proximal policy optimization (PPO), and standard methods, show the effectiveness of using learning-based algorithms in terms of minimizing the packet loss and the energy consumption.