Deep Reinforcement Learning for Collaborative Offloading in Heterogeneous Edge Networks

Mobile Edge Computing (MEC) has been envisioned as an emerging paradigm to handle the overwhelming explosion of mobile applications and services, by allowing edge devices (EDs) to offload their computationally-intensive tasks to heterogeneous MEC servers. Most of the existing works focus mostly on a centralized agent or an independent multi-agent setting which cannot work well in distributed edge networks with heterogeneous computation tasks. This paper considers a more realistic setting consisting of multiple cooperative EDs and multiple MEC servers in heterogeneous edge networks (HENs). We propose a new collaborative offloading framework in a HEN where each ED acts as an intelligent agent to make offloading decisions collaboratively, aiming to achieve the optimal system utility. To this end, we formulate the collaborative offloading problem as a Markov game which is then solved by a novel multi-agent deep reinforcement learning (MADRL) approach based on a multi-agent deep deterministic policy gradient (MA-DDPG) algorithm. Numerical simulations with real-life mobile wireless datasets show that the proposed cooperative multi-agent scheme can improve the system utility by 43.6% compared to the non-cooperative offloading schemes.