A Multi-update Deep Reinforcement Learning Algorithm for Edge Computing Service Offloading

By pushing computing functionalities to network edges, backhaul network bandwidth is saved and various latency requirements are met, providing support for diverse computation-intensive and delay-sensitive multimedia services. Due to the limited capabilities of edge nodes, it is very important to decide which services should be provided locally. This paper investigates the cloud-edge service offloading problem. Different from prior works which only give the proportion of computation offloading with constraint of computing capacity, we also take the storage space into account and determine the computing status of each service. We formulate the problem as a Markov decision process whose goal is to maximize the long-term average reduction of delay. The problem is hard to be solved with traditional methods because of the extremely large action space and lack of information about transition probability. Instead, this paper proposes an innovative deep reinforcement learning method to solve it. The proposed multi-update reinforcement learning algorithm introduces a novel exploration strategy and update method, which reduce dramatically the size of the action space. Extensive simulation-based testing shows that the proposed algorithm has fast convergence and improves the system performance more than other three alternative solutions do.