Scalable and Efficient URLLC Resource Allocation Based on Offline Graph Reinforcement Learning

Deep reinforcement learning (RL) has emerged as a transformative technology for addressing intricate radio resource allocation problems in both 5G and 6G networks. Nevertheless, during the training phases of RL-based resource allocation algorithms, online RL methods introduce potential risks as they necessitate continuous interaction with the environment. Motivated by the fact that the historical data of the resource allocation process in the base station (BS) reveals the operation rules of the mobile network dynamics, this paper proposes a novel data-driven framework for resource allocation optimization, termed Offline Graph Reinforcement Learning (OGRL). Different from existing intelligent resource allocation schemes, which acquire policies by interacting either with wireless network simulators or real-world networks, the proposed OGRL framework exclusively leverages historical data from BS to train a high-performance resource allocation policy. Furthermore, by harnessing the capabilities of the graph neural networks (GNNs) at processing intricate data structures, OGRL can adeptly manage dynamic network environments characterized by continuously changing active users. Extensive experimental results substantiate that, following training with historical data, the proposed OGRL attains Quality of Service (QoS) and packet loss rates comparable to those of the best online algorithms. Furthermore, OGRL exhibits excellent scalability and generalization capabilities.