Multi-UAV Cooperative Search in Partially Observable Low-Altitude Environments Based on Deep Reinforcement Learning

Multi-Unmanned Aerial Vehicle (Multi-UAV) cooperative search represents a cutting-edge research direction in the field of unmanned aerial vehicle applications. The use of multi-UAV systems for low-altitude target search and area surveillance has become an effective means of enhancing security capabilities. In practical scenarios, UAVs rely on onboard sensors to acquire environmental information; however, due to the limited perceptual range of these sensors, their observation capabilities are inherently local and constrained. This paper investigates the problem of multi-UAV cooperative search in partially observable low-altitude environments, where each UAV possesses a circular sensing range with a finite radius. Target location information is only obtained when a target enters the field of view of any UAV. The objective is to achieve cooperative search and sustain continuous surveillance while ensuring safety among UAVs and with the environment. To address this challenge, we propose a novel multi-agent deep reinforcement learning (MADRL) algorithm named Normalizing Graph Attention Soft Actor-Critic (NGASAC). This algorithm integrates a normalizing flow (NL) layer and a multi-head graph attention network (MHGAT). The normalizing flow technique maps traditional Gaussian sampling to a more complex action distribution, thereby enhancing the expressiveness and flexibility of the policy. Simultaneously, by constructing a multi-head graph attention network that captures “obstacle–target” relationships, the algorithm improves the UAVs’ ability to learn and reason about complex spatial topologies, leading to significantly better performance in cooperative search and stable surveillance of hidden targets. Simulation results demonstrate that the NGASAC algorithm markedly outperforms baseline methods such as Multi-Agent Soft Actor-Critic (MASAC), Multi-Agent Proximal Policy Optimization (MAPPO), and Multi-Agent Deep Deterministic Policy Gradient (MADDPG) across multiple evaluation metrics, including success rate, task time, and obstacle avoidance capability. Furthermore, it exhibits strong generalization performance and robustness.