Autonomous Tracking Using a Swarm of UAVs: A Constrained Multi-Agent Reinforcement Learning Approach

In this paper, we aim to design an autonomous tracking system for a swarm of unmanned aerial vehicles (UAVs) to localize a radio frequency (RF) mobile target. In the system, UAVs equipped with omnidirectional received signal strength (RSS) sensors can cooperatively search the target with a specified tracking accuracy. To achieve fast localization and tracking in the highly dynamic channel environment (e.g., time-varying transmit power and intermittent signal), we formulate a flight decision problem as a constrained Markov decision process (CMDP) with the main objective of avoiding redundant UAV flight path. Then, we propose an enhanced multi-agent reinforcement learning to coordinate multiple UAVs performing real-time target tracking. The core of the proposed scheme is a feedback control system that takes into account the uncertainty of the channel estimate. We prove that the proposed algorithm can converge to the optimal decision. Our simulation results show that the proposed scheme outperforms standard Q-learning and multi-agent Q-learning algorithms in terms of searching time and successful localization probability.