Dense Multiagent Reinforcement Learning Aided Multi-UAV Information Coverage for Vehicular Networks

With the rapid development of wireless communication networks, UAVs serving as base stations are increasingly being applied in various scenarios which not only include edge computation and task offloading, but also involve emergency communication, vehicular network enhancement, etc. In order to enhance the utility of UAV base stations' allocation and deployment, a series of algorithms have been proposed, utilizing heuristic methods, learning-based algorithms or optimization approaches. However, it is intractable for current algorithms to handle the exponential computation increment with UAV base stations increasing, and complicated application scenarios with high dynamic demands. To solve the above issues, we formulate a decision problem with a long sequence to optimize the deployment of multi-UAV base stations for maximizing vehicular networks' communication coverage ratio, which needs to be subject to co-constraints consisting of moving velocity, energy consumption and communication coverage radius. To solve this optimization problem, we creatively propose an algorithm named dense multi-agent reinforcement learning (DMARL), which is under the dual-layer nested decision-making framework, centralized training with decentralized deployment, and accelerates training by only collecting critical states into the dense sampling buffer. To prove our proposed algorithm's effectiveness and generalization ability, we conduct experimental simulations in scenarios with different scales. Corresponding results have been provided to verify our algorithm's superiority in training efficiency and performance metrics, including coverage ratio and energy consumption, compared with other algorithms.