Leveraging Multi-Object Tracking in Vision-based Target Following for Unmanned Aerial Vehicles

This paper presents an autonomous Vision-based Mobile Target Tracking and Following system designed for Unmanned Aerial Vehicles, enabling them to effectively track and pursue mobile targets on the ground. Rather than the conventional Single-Object Tracking strategy, this paper explores the capabilities of recent advances of Multi-Object Tracking methods in a target following scenario. We leverage cutting-edge detection and tracking capabilities by integrating You Only Look Once (YOLOv8) with the MOT BoT-SORT, enabling the extraction of multi-target information. Complementing visual data, we incorporate a depth sensing module to enhance distance estimation when feasible. For target following, a 3D flight control system is proposed, capable of reacting to the targets speed and direction changes, while maintaining line-of-sight. The full system is deployed using the MRS UAV System 1 in a realistic simulation capable of future real-world deployment. Results show that the system achieves precise target tracking and following, resilient to partial and full occlusions in dynamic environments, effectively distinguishing the followed target from bystanders.