Multi-UAV Escape Target Search: A Multi-Agent Reinforce-ment Learning Method

The multi-UAV target search problem is crucial in the field of autonomous Unmanned Aerial Vehicle (UAV) decision-making. The algorithm design of Multi-agent Reinforcement Learning (MARL) methods has become integral to research on multi-UAV target search, due to its adaptability to the rapid online decision-making required by UAVs in complex, uncertain environments. In non-cooperative target search scenarios, targets may have the ability to escape, complicating UAVs' search efforts and hindering the convergence of the MARL algorithm training. This paper investigates the multi-UAV target search problem in scenarios involving static obstacles and dynamic escape targets, modeling the problem within the framework of Decentralized Partially Observable Markov Decision Process. Based on this model, a Spatio-Temporal Efficient Exploration network and a Global Convolutional Local Ascent mechanism are proposed. Subsequently,we introduce a Multi-UAV Escape Target Search Algorithm Based on MAPPO(ETS-MAPPO) for addressing the escape target search difficulty problem. Simulation results demonstrate that the ETS-MAPPO algorithm outperforms five classical MARL algorithms in terms of the number of target searches,the area coverage rate, and other metrics.