Dynamic Spectrum Anti-Jamming in Broadband Communications: A Hierarchical Deep Reinforcement Learning Approach

In this letter, the frequency selection problem in jamming environment with large number of optional frequencies is investigated. With numerous optional actions in the wider frequency band scenario, most of existing anti-jamming methods will become ineffective, since the convergence time and computational complexity will grow exponentially with the number of actions. To cope with the above challenge, a novel hierarchical deep reinforcement learning algorithm which does not need to know the jamming patterns and channel model is proposed. The proposed algorithm divides the frequency selection problem in the broadband into two steps via two subnetworks: Firstly, the frequency band is selected by the band selection network, and then the specific frequency is selected in this frequency band by the frequency selection network. Simulation results show that the proposed algorithm avoids multiple different jammings effectively and achieves satisfactory throughput with less calculation.