A Stackelberg Game-Theoretic Hierarchical DRL Framework for RIS-Assisted ISAC Under Mobility

Integrated sensing and communication (ISAC) is a key enabler of next-generation wireless networks, where communication, sensing, and mobility-aware beam management must be jointly optimized under dynamic and imperfectly observed conditions. This paper proposes a game-theoretic hierarchical deep reinforcement learning framework for RIS-assisted ISAC (GT–HIDRL–ISAC), which formulates the coupled beamforming, RIS, sensing, and power-control problem as a Stackelberg bilevel optimization. A master agent jointly controls BS beamforming, RIS phase configuration, and sensing level, while a slave agent performs user-specific power control as an approximate best response. Hierarchical update timescales and centralized critics are employed to improve the learning process, while EKF-based mobility tracking accounts for imperfect sensing under Rician LoS/NLoS channels. Theoretical analysis establishes strong Stackelberg equilibrium (SSE) existence under standard regularity conditions and quantifies the effect of slave approximation errors on the master Stackelberg gradient. Simulation results show that GT–HIDRL–ISAC achieves approximately 0.82 bits/J median energy efficiency, compared with 0.795 bits/J for the no-Stackelberg ablation and 0.76 bits/J for Flat MAAG. It also achieves a median rate of 5.19 Mbps and a median tracking error of 1.00 m, corresponding to approximately 12.3% higher rate and 38.3% lower tracking error than Flat MAAG. Additional evaluations against TD3, SAC, AO–SCA, and AO–SDR demonstrate consistent gains across the considered channel, sensing, distance, multi-user, and RIS-size configurations.