Joint Energy-Efficient and Throughput Optimization in Large-Scale Mobile Networks via Safe Hierarchical MARL

The rapid advancement of 5G networks has significantly increased the demand for high throughput and low latency, particularly for enhanced Mobile Broadband (eMBB) services. Optimizing user rates in large-scale base station (BS) collaborative scenarios is crucial, ensuring high throughput and low energy consumption while meeting the needs of users. The optimization process faces two key challenges: how to efficiently explore and utilize the vast action space for collaborative decision-making in large-scale networks with numerous BSs, and how to ensure optimal decision-making across numerous parameters while satisfying constraints. To tackle these challenges, we propose a novel solution that integrates a generative AI-powered Digital Twin platform with a multi-agent reinforcement learning (MARL) decision algorithm. An important tool is the high-fidelity DT-SimNet platform, enabling seamless integration with AI optimizers. Moreover, we propose a novel algorithm called Safe Hierarchical Multi-Agent Proximal Policy Optimization with Adaptive Prediction mechanism (Safe HMAPPO-AP). This algorithm integrates Safe MARL to ensure decision-making remains within constraint boundaries and incorporates a hierarchical structure to decompose the action space. Moreover, it leverages Graph Neural Networks (GNNs) for efficient feature extraction and employs the Adaptive Prediction mechanism to accelerate learning and improve network performance. Comprehensive experiments using the DT-SimNet platform demonstrate the superior performance of the proposed algorithm compared to competing methods. Within the target area, Safe HMAPPO-AP achieved a 6.73% improvement in average throughput for regular users and an 11.48% increase in system utility, while meeting demands of newly added eMBB users.