Federated Deep Reinforcement Learning-Based Spectrum Access Algorithm With Warranty Contract in Intelligent Transportation Systems

Cognitive radio (CR) provides an effective solution to meet the huge bandwidth requirements in intelligent transportation systems (ITS), which enables secondary users (SUs) to access the idle spectrum of the primary users (PUs). However, the high mobility of users and real-time service requirements result in the additional transmission collisions and interference, which degrades the spectrum access rate and the quality of service (QoS) of users in ITS. This paper proposes a spectrum access algorithm (Feilin) based on federated deep reinforcement learning (FDRL) to improve spectrum access rate, which maximizes the QoS reward function with considering the hybrid benefits of delay, transmission power and utility of SUs. To guarantee the utility of SUs, the warranty contract is designed for SUs to obtain compensation for data transmission failure, which promotes SUs to compete for more spectrum resources. To meet the real-time requirements and improve QoS in ITS, a spectrum access model called FDQN-W is proposed based on federated deep Q-network (DQN), which adopts the asynchronous federated weighted learning algorithm (AFWLA) to share and update the weights of DQN in multiple agents to decrease time cost and accelerate the convergence. Detailed simulation results show that, in the multiuser scenario, compared with the existing methods, the proposed algorithm Feilin increases the spectrum access success rate by 15.1%, and reduces the collision rate with SUs and the collision rate with PUs by 46.4% and 6.8%, respectively.