Distributed Federated Deep Reinforcement Learning Based Trajectory Optimization for Air-Ground Cooperative Emergency Networks

The air-ground cooperative emergency networks can assist with the rapid reconstruction of communication in the disaster area, where unmanned aerial vehicles (UAVs) and unmanned ground vehicles (UGVs) are deployed as base stations. The trajectory optimization of emergency base stations is of vital importance to the communication performance, which is related to the timeliness and effectiveness of rescue. In this paper, federated multi-agent deep deterministic policy gradient (F-MADDPG) based trajectory optimization algorithm is proposed to maximize the average spectrum efficiency. Specifically, the property of MADDPG is inherited to jointly control of multiple vehicles and federated averaging (FA) is utilized to eliminate the isolation of data to accelerate the convergence. Distributed F-MADDPG (DF-MADDPG) is further designed to reduce the communication overhead with a distributed architecture. The simulation results indicate that the proposed F-MADDPG and DF-MADDPG based algorithms significantly outperform the existing trajectory optimization algorithms, in terms of the average spectrum efficiency and the speed of convergence.