Enhanced underwater object tracking in sonar images using deep learning and vehicle motion data

Abstract Object tracking is a crucial aspect for maritime operations and navigation for underwater vehicles. Forward-looking sonar images, which provide high-resolution views of the underwater environment, enable the identification and classification of objects. However, the interpretation of these sonar images can be challenging due to feature distortion, reverberation and environmental noise, complicating object detection. To address these issues, in this paper we propose an enhanced method for underwater multi-object tracking that utilizes deep learning techniques and vehicle motion data. Our object detection framework encompasses range-setting, sliding window preprocessing, and deep learning-based multi-object detection that leverages visual features from sonar images. This process is further enhanced by integrating vehicle motion data collected from navigation devices and sensors. We introduce a combinatory approach for multi-object tracking of static objects, which integrates sonar images with vehicle motion information. The algorithm determines object locations by calculating trajectories in the sonar images using navigation and vehicle movement data and is updated with the object detection algorithm based on visual features in the sonar image. The inclusion of vehicle motion data significantly enhances the precision, recall and processing time of object tracking, ensuring continuous tracks even when objects are temporarily not seen in sonar images. Therefore, our proposed combinatory procedure enhances the algorithm’s robustness against common sonar image challenges, enabling reliable multi-object tracking of static objects in sonar images and supporting subsequent navigational tasks.