When Learning Meets Dynamics: Distributed User Connectivity Maximization in AAV-Based Communication Networks

Distributed network management over Autonomous Aerial Vehicle (AAV) based communication networks (ACNs) has attracted increasing research attention. In this work, we study a distributed user connectivity maximization problem in a ACN. The work features a horizontal study over different levels of information exchange during the distributed iteration and a consideration of dynamic AAV set and user distributions, which are not well addressed in existing work. Specifically, the studied problem is first formulated into a time-coupled mixed-integer non-convex optimization problem. The problem aims to maximize the accumulated number of served users in a slotted time horizon by optimizing AAV trajectories and determining user association. A heuristic two-stage AAV-user association policy is proposed to faster determine the user connectivity. To tackle the NP-hard problem in a scalable manner, the distributed user connectivity maximization algorithm 1 (DUCM-1) is proposed under a multi-agent deep Q learning (MA-DQL) framework. DUCM-1 emphasizes on the design of different information exchange levels and evaluating how they impact the learning convergence with stationary and dynamic user distribution. To comply with the AAV dynamics, DUCM-2 algorithm is devoted to autonomously handling arbitrary quit’s and join-in’s of AAVs in a considered time horizon. Extensive simulations are conducted i) to conclude that exchanging state information with a deliberated task-specific reward function design yields the best convergence performance, and ii) to show the efficacy and robustness of DUCM-2 against the dynamics.