Intelligent Resource Management Based on Reinforcement Learning for Ultra-Reliable and Low-Latency IoV Communication Networks

Internet of Vehicles (IoV) has attracted much interest recently due to its ubiquitous message exchange and content sharing among smart vehicles with the development of the mobile communication and computation technologies. In this paper, we investigate the policy for jointly communication mode selection, resource block assignment, and power control in device-to-device-enabled vehicle-to-vehicle (V2V) based IoV communication networks with the purpose of guaranteeing the strict ultra-reliable and low latency communications requirements of V2V links while maximizing the sum capacity of vehicle-to-infrastructure links. Considering the unknown environments dynamics as well as the continuous-valued state and action space in IoV networks, we exploit a decentralized actor-critic reinforcement learning model with a new reward function to learn the policy by interacting with the environment. Moreover, we propose an efficient transfer actor-critic learning (ETAC) approach to effectively enhance the learning efficiency and improve the learning convergence speed, in order to support reliable and delay-sensitive vehicular services in IoV networks. Simulation results show that the proposed ETAC approach can effectively reduce the generated interference in IoV networks and ensure the latency and reliability requirements of V2V link, as well as achieve the fast convergence speed and high convergence stability, compared with other existing approaches.