DRL-Powered Sum-Rate Maximization Design for IRS-Assisted Full Duplex Cluster NOMA WPCNs

This paper tackles the complex problem of maximizing the ergodic sum-rate in intelligent reflecting surface (IRS)-assisted full-duplex (FD) cluster non-orthogonal multiple access wireless-powered communication networks. The primary challenge lies in the joint optimization of hybrid access point (HAP) beamforming vectors, IRS phase shifts, and uplink power allocation for sensors, while considering practical constraints— including channel uncertainties, a nonlinear energy harvesting model, self-interference at both the HAP and sensors due to FD transmission, the rank-one constraint of the IRS, and the limited buffer capacity of the sensors. These factors result in an NP-hard optimization problem that conventional approaches cannot solve efficiently. To address this, we propose utilizing a deep reinforcement learning framework based on the twin delayed deep deterministic policy gradient (TD3) algorithm. Our results show that TD3 improves the system’s average sum-rate by up to 12% and 48% over the deep deterministic policy gradient (DDPG) and random allocation benchmarks, respectively. The results also show that both TD3 and DDPG have similar computational complexity. Thus, TD3 achieves a favorable balance between performance and computational complexity, making it a scalable and practical solution for next-generation networks where spectrum efficiency and throughput maximization are critical in hyper-dense environments.