Maritime vision datasets for autonomous navigation: A comparative analysis

Artificial intelligence is becoming an increasingly essential component in many areas, with notable advancements being made in the field of maritime computer vision. The employed deep learning models require substantial quantities of high-quality training data that are specifically tailored to the tasks for which they are being applied in the maritime domain. Training autonomous navigation systems for unmanned surface vehicles has been significantly enhanced by using extensive visual datasets captured through high-quality cameras, enabling these systems to learn from diverse environmental scenarios and improve the decision-making accuracy. However, the identification of suitable publicly accessible maritime vision datasets is challenging, and there is currently no broad overview of datasets that have been specifically designed for computer vision tasks related to unmanned surface vehicles in the maritime domain. This survey addresses the identified research gap by providing a comprehensive and systematic overview of open-source vision datasets containing ships, taking into account the specific task, the surrounding environment, and additional available data, such as infrared images or time series information. It is our aim to assist new researchers in the field of maritime computer vision to gain a rapid overview and facilitate initial access to this domain, enabling them to identify the most suitable dataset for their particular task. Highlights The review provides the most comprehensive and up-to-date survey of 25 open-source maritime vision datasets published between 2015 and October 2024, with a focus on autonomous navigation. It introduces a novel set of systematic criteria for dataset analysis tailored to autonomous navigation, evaluating task suitability, environmental diversity, and annotation quality. It reveals a positive trend in the annual publication of maritime datasets and identifies the most influential datasets through a citation analysis. It offers specific, novel recommendations for autonomous navigation tasks, highlighting the LaRS and MVDD13 datasets for their suitability in advanced applications and robustness in diverse conditions.