Joint Offloading and Resource Allocation in Mobile Edge Computing Systems: An Actor-Critic Approach

Offloading computationally intensive tasks from user equipments (UEs) to mobile edge computing (MEC) servers is a promising technique to boost up the computational capacity of UEs. However, MEC will incur extra energy consumption and time delays, which motivates the deployment of energy harvesting (EH) small cell networks with MEC in mobile networks. Due to the complexity of such networks, it is challenging to effectively allocate resources for UEs. In this paper, we investigate the offloading decision, wireless and computational resources allocation problem in energy harvesting (EH) small cell networks with MEC. Different from existing literatures, our research focuses on improving mobile operators' revenue by maximizing the amount of the offloaded tasks while decreasing the energy expenditure and time-delays. Besides, queues are created at the MEC server side to store the un-executed tasks in a time slot, which is used as a punishment in our utility function to avoid serious delay. Considering the varying lengths of queues, the states of EH-batteries of small base stations (SBSs) and down-link channels, the above problem is modeled as a Markov decision process (MDP). Since the states and actions in the MDP are infinite, an online and on-policy actor-critic with eligibility traces algorithm is proposed to resolve the problem. Simulation results show the proposed algorithm has superior performances compared with the policy-gradient algorithm and Q-learning.