Deep Reinforcement Learning-Based Modulation and Coding Scheme Selection in Cognitive Heterogeneous Networks

We consider a cognitive heterogeneous network (HetNet), in which multiple pairs of secondary users adopt sensing-based approaches to coexist with a pair of primary users on a certain spectrum band. Due to imperfect spectrum sensing, secondary transmitters (STs) may cause interference to the primary receiver (PR) and make it difficult for the PR to select a proper modulation and/or coding scheme (MCS). To deal with this issue, we exploit deep reinforcement learning (DRL) and propose an intelligent MCS selection algorithm for the primary transmission. To reduce the system overhead caused by the MCS switchings, we further introduce a switching cost factor in the proposed algorithm. The simulation results show that the primary transmission rate of the proposed algorithm without the switching cost factor is 90% ~ 100% of the optimal MCS selection scheme, which assumes that the interference from the STs is perfectly known at the PR as prior information, is 30% higher than that of the upper confidence bandit (UCB) algorithm, and is 100% higher than that of the signal-to-noise ratio (SNR)-based algorithm. Meanwhile, the proposed algorithm with the switching cost factor can achieve a higher primary transmission rate than those of the benchmark algorithms without increasing system overheads.