UAV Communication Against Intelligent Jamming: A Stackelberg Game Approach With Federated Reinforcement Learning

This paper proposes a novel anti-intelligent jamming framework for unmanned aerial vehicle (UAV) networks. Multiple UAV-to-UAV communication pairs aim to maximize their sum rates with minimal power consumption, where each UAV adaptively adjusts its transmit channel and power in a distributed way to avoid intelligent jamming and co-channel interference. A ground jammer attempts to disrupt the communication quality of the UAV network by adaptively altering its jamming channel and power. We model the anti-jamming problem as a stochastic Stackelberg game, where the intelligent jammer is the leader and the UAV pairs are the followers. Considering that both parties are unwilling to share their utility functions and transmission policies, we propose reinforcement learning (RL) algorithms to solve the best response policies of each agent in the game. We adopt deep Q network (DQN) algorithm to decide the jamming policy at the jammer and propose a decentralized federated learning-assisted DQN algorithm to determine the collaborative anti-jamming policies at the UAV pairs. Simulation results demonstrate that the performance of the proposed algorithm achieves an improvement of 23.3% in anti-jamming performance compared with the independent DQN algorithm.