HideAndSeg: an AI-driven approach to monitoring benthic octopuses in the wild

Analyzing the behavior and body patterns of octopuses in nature is challenging due to their camouflage capabilities, changes in skin texture and color, body deformations, and frequent occlusions. Underwater lighting and turbidity further complicate these challenges. Addressing the lack of large-scale annotated datasets, we introduce HideAndSeg, a minimally supervised, AI-based tool for segmenting octopus videos. HideAndSeg integrates SAM2 with a custom-trained YOLOv11 object detector. First, the user provides point coordinates to generate the initial segmentation masks with SAM2. These masks serve as training data for the YOLO model. Our approach automates the pipeline by providing a bounding-box prompt to SAM2, eliminating manual intervention. We introduce two unsupervised metrics – temporal consistency and new component count – to quantitatively evaluate segmentation quality and guide mask refinement in the absence of real-world information used to train, validate, and test AI models. HideAndSeg reduces segmentation noise compared to the manually prompted approach. Our method can re-identify and segment the octopus even after complete occlusion, a scenario in which the manually prompted model fails. By reducing manual analysis in real-world scenarios, this work provides a practical tool that enables more efficient behavioral studies of wild octopuses.