Open RAN Slicing for MVNOs With Deep Reinforcement Learning

As 5G networks continue to be deployed and 6G networks begin to be envisioned, mobile network operators (MNOs) are embarking on a revolutionary transformation of the way they manage their networks. Various technology bricks are currently considered paramount in this transformation, including radio access network (RAN) slicing. The concept of an open radio access network (Open RAN) promises to provide more flexibility to support RAN slicing. However, RAN slicing in an O-RAN architecture raises a major challenge in achieving efficient resource sharing among slices, due to the diverse and permanent changes in RAN slices' QoS requirements. To overcome this challenge in a RAN environment involving an MNO and multiple mobile virtual network operators (MVNOs), we propose a two-level RAN slicing mechanism. The first level is executed on a long time-scale to allocate radio resources from the MNO to MVNOs while the second level is executed on a shorter time-scale to allocate MVNO resources to users. This mechanism improves the performance of the RAN slicing operation by enabling users to obtain the required resources as quickly as possible and with a high level of granularity. We formulate the two-level problem as two mathematical optimization problems and we study their NP hardness. To efficiently solve the two-level problem, we first propose a game-theoretic solution to solve the first-level resource allocation problem using a matching algorithm. Next, we propose a deep reinforcement learning (DRL) algorithm that uses the double deep Q -network procedure to solve the second-level resource allocation problem. The two proposed algorithms are coupled such that the DRL algorithm uses the solution obtained using the game-theoretic matching algorithm. We show through extensive simulations that the proposed two-level solution outperforms the current state-of-the-art solutions and achieves efficient performance.