A Constrained Deep Reinforcement Learning Framework for Beamforming Optimization With QoS Guarantees in RIS‐Assisted Symbiotic Radio Networks

Reconfigurable intelligent surface (RIS) assisted symbiotic radio communication (SRC) networks are considered a key solution to address the challenges of massive Internet of Things connectivity. There, the resource optimization problems are always NP‐hard and formulated with time varying parameters. Learning via deep neural network has proven to be an effective way to encapsulate the optimized solution as a function of the optimization problem. However, the learning framework is challenged by meeting complex communication constraints involved. To handle this issue and accommodate the time varying SRC environment, this work proposes a novel deep reinforcement learning framework to solve the classical passive beamforming optimization problem. System sum‐rate is maximized while ensuring strict communication requirements. The formulated constrained Markov decision process is transformed into a Lagrange dual problem. The deep deterministic policy gradient framework is adopted introducing a monitor module to approximate Lagrange multipliers. The actor and monitor networks are trained using the derived Lagrangian‐related gradients, ensuring constraint satisfaction during the training. Simulation results demonstrate that the proposed method achieves the performance upper bound, significantly improving RIS transmission rate and system sum‐rate compared to the counterparts. It also demonstrates adaptability to environmental dynamics, computational efficiency, reduced transmission resource usage, and lower probability of violating communication requirements.