Hierarchical Role-Based Multi-Agent Reinforcement Learning for UHF Radiation Source Localization with Heterogeneous UAV Swarms

With the continuous proliferation of radio frequency devices, electromagnetic environments in various regions are becoming increasingly complex. Effective monitoring of the electromagnetic environment and identification of interference sources have thus become critical tasks for maintaining order in the electromagnetic spectrum. In recent years, rapid advances in UAV technology have spurred exploration of UAV-based electromagnetic spectrum monitoring as a novel approach. However, the limited payload capacity and endurance of UAVs constrain their monitoring capabilities. To address these challenges, we propose HMUDRL, a distributed heterogeneous multi-agent deep reinforcement learning algorithm. By leveraging cooperative operation between cluster-head UAVs (CH) and cluster-monitoring UAVs (CM) within a heterogeneous UAV swarm, HMUDRL enables high-precision detection and wide-area localization of UHF radiation source. Furthermore, we integrate a minimum-gap localization algorithm that exploits the spatial distribution of multiple CM to accurately pinpoint anomalous radiation sources. Simulation results validate the effectiveness of HMUDRL: in the later stages of training, the success rate of localizing target radiation sources converges to 96.1%, representing an average improvement of 1.8% over baseline algorithms; localization accuracy, measured by root mean square error (RMSE), is enhanced by approximately 87.3% compared to baselines; and communication overhead is reduced by more than 80% relative to homogeneous architectures. These results demonstrate that HMUDRL effectively addresses the challenges of data transmission control and sensing-localization performance faced by UAVs in UHF spectrum monitoring.