Reinforcement Learning-based Resource Allocation in Fog RAN for IoT with Heterogeneous Latency Requirements

In light of the quick proliferation of Internet of things (IoT) devices and, fog radio access network (Fog-RAN) has been recently proposed for generation (5G) wireless communications to assure the requirements of-reliable low-latency communication (URLLC) for the IoT applications which accommodate large delays. Hence, fog nodes (FNs) are equipped with, signal processing and storage capabilities to extend the inherent and services of the cloud to the edge. We consider the problem of allocating the FN's limited resources to the IoT applications of latency requirements. For each access request from an IoT user, FN needs to decide whether to serve it locally utilizing its own resources to refer it to the cloud to conserve its valuable resources for future users potentially higher utility to the system (i.e., lower latency requirement). formulate the Fog-RAN resource allocation problem in the form of a Markov process (MDP), and employ several reinforcement learning (RL) methods, Q-learning, SARSA, Expected SARSA, and Monte Carlo, for solving the MDP by learning the optimum decision-making policies. We verify the and adaptivity of the RL methods and compare it with the of a fixed-threshold-based algorithm. Extensive simulation results 19 IoT environments of heterogeneous latency requirements that RL methods always achieve the best possible performance of the IoT environment.