Continuous control deep reinforcement learning for joint resource optimization in STAR-RIS assisted mmWave-NOMA systems with hardware impairments

This paper proposes a continuous-control deep reinforcement learning (DRL) framework for the joint optimization of power allocation, hybrid beamforming, and phase-shift configuration in simultaneously transmitting and reflecting reconfigurable intelligent surface (STAR-RIS)-assisted millimeter-wave non-orthogonal multiple access (mmWave-NOMA) networks. The proposed system supports heterogeneous services, including enhanced mobile broadband (eMBB) and ultra-reliable low-latency communication (uRLLC), by maximizing the minimum signal-to-interference-plus-noise ratio (SINR) among users while satisfying quality-of-service requirements. Practical hardware impairments, including imperfect successive interference cancellation (SIC) and in-phase and quadrature imbalance (IQI), are explicitly incorporated into the system model to capture residual interference and transceiver distortions. A continuous-control DRL framework based on Softmax Deep Double Deterministic Policy Gradients (SD3) is developed to jointly optimize the network resources under these non-ideal conditions. Extensive simulation results demonstrate that the proposed STAR-RIS-assisted SD3 framework consistently outperforms conventional RIS-based schemes and benchmark DRL algorithms in terms of achievable sum rate and convergence performance. Furthermore, robustness evaluations under imperfect channel state information (CSI), varying hardware impairment levels, and different Rician fading conditions show that the proposed approach maintains superior performance despite channel uncertainty and practical transceiver impairments, highlighting its effectiveness for next-generation mmWave communication systems.