Reinforcement learning-based joint beam selection and network slicing for 5G-advanced O-RAN

Open Radio Access Network (O-RAN) makes 5G-Advanced networks more flexible through software-based components called xApps and rApps. However, rapid variation in channel conditions make beam selection and resource allocation difficult. This paper proposes a reinforcement learning (RL) framework to address these challenges. It works across the Near-Real-Time and Non-Real-Time RIC platforms. The framework focuses on two main tasks. The first selects the best beam for each user. The second allocates bandwidth among eMBB (enhanced Mobile Broadband), URLLC (Ultra-Reliable Low Latency Communications), and mMTC (massive Machine Type Communications) slices. Minimum rate requirements are also considered. A multi-objective reward function guides the learning process. Two tabular Q-learning xApps are developed. The beam-selection xApp uses 864 states, while the slicing xApp uses only 9 states. The proposed method was tested through 30 independent Monte Carlo trials. Results include 95% confidence intervals. The study also compares the method with greedy and random approaches. An ablation study was conducted to understand the contribution of each xApp. The slicing xApp performed well. It achieved an average throughput of 1709.2 Mbps, compared with 1639.4 Mbps for the greedy method. In contrast, the beam-selection xApp showed limited improvement. It achieved 700.4 Mbps, while random selection achieved 680.4 Mbps. The large state space made tabular Q-learning less effective. A handover hysteresis mechanism reduced handover events by 23.8%, but aggregate throughput decreased by 10.5%. This shows the trade-off between stable connections and network performance. Overall, tabular RL appears suitable for network slicing. For beam selection, channel-aware greedy methods may be more practical. Deep RL could be explored in future work for more complex beam-selection problems.