UAV-PanoSeg: A large-scale dual-modality benchmark for UAV-based panoptic segmentation

We present a novel UAV-based dataset to advance research in panoptic segmentation by integrating object recognition within the broader context of scene understanding. The dataset is constructed from aerial infrared and visible-spectrum imagery and targets complex outdoor environments where multiple objects coexist under dynamically varying conditions, including changes in altitude, object scale, spatial density, and background composition. Each image is annotated with instance-level segmentations and corresponding semantic labels, enabling precise object localization and comprehensive scene parsing. The dataset focuses on objects relevant to UAV-based surveillance and monitoring applications, comprising 2,079,446 finely annotated instances captured across challenging viewpoints and scales. The annotation process follows a carefully designed multi-stage pipeline incorporating validation rules, consistency constraints, and iterative manual refinement to ensure high-quality panoptic annotations. We provide a detailed statistical analysis of the dataset and benchmark it against existing standards, highlighting its unique challenges and contributions. The dataset is intended to support real-world UAV applications, including search and rescue, surveillance, traffic monitoring, and disaster response in diverse environmental conditions.