Annotation strategy trade-offs for deep learning-based underwater fish detection

Camera-based ecosystem monitoring can provide data which is rich with insights into species abundance and behavior, but this information is commonly locked behind long analysis times requiring trained experts to identify taxa and habitat conditions. Methods using deep learning offer the ability to process these data more efficiently than relying entirely on human annotators, but commonly require large training datasets to adapt architectures to ecological tasks. In this study, we compare two potential annotation strategies for building small-medium size datasets (<1500 annotations per class) for underwater faunal monitoring with computer vision. We also explore the effects of architecture choice between You Only Look Once (YOLO) versions 8, 9, 10 and 11. When comparing a specialist annotation strategy to a general one, we found that the former offers comparable performance when sufficient annotations are used in training (2.85% difference in F1 scores in validation). When the number of examples of individual classes is limited, our results showed general detection models having better class-wise performance (mean 21.35% improvement in F1 score in validation). For model selection, we did not find significant differences in performance between YOLO versions tested on our dataset, but propose YOLOv9 as a frontrunner based on a weighted ranking incorporating accuracy and various training and efficiency metrics. The study suggests both types of object detectors have their place in ecosystem monitoring, with specialist one favoring projects with sufficient annotations for a narrow group of interest, and general ones introducing more versatility across classes and annotation levels.