IMM-DeepSort: An Adaptive Multi-Model Kalman Framework for Robust Multi-Fish Tracking in Underwater Environments

Multi-object tracking (MOT) is a critical task in computer vision, with widespread applications in intelligent surveillance, behavior analysis, autonomous navigation, and marine ecological monitoring. In particular, accurate tracking of underwater fish plays a significant role in scientific fishery management, biodiversity assessment, and behavioral analysis of marine species. However, MOT remains particularly challenging due to low visibility, frequent occlusions, and the highly non-linear, burst-like motion of fish. To address these challenges, this paper proposes an improved tracking framework that integrates Interacting Multiple Model Kalman Filtering (IMM-KF) into DeepSORT, forming a self-adaptive multi-object tracking algorithm tailored for underwater fish tracking. First, a lightweight YOLOv8n (You Only Look Once v8 nano) detector is employed for target localization, chosen for its balance between detection accuracy and real-time efficiency in resource-constrained underwater scenarios. The tracking stage incorporates two complementary motion models—Constant Velocity (CV) for regular cruising and Constant Acceleration (CA) for rapid burst swimming. The IMM mechanism dynamically evaluates the posterior probability of each model given the observations, adaptively selecting and fusing predictions to maintain both responsiveness and stability. The proposed method is evaluated on a real-world underwater fish dataset collected from the East China Sea, comprising 19 species of marine fish annotated in YOLO format. Experimental results show that the IMM-DeepSORT framework outperforms the original DeepSORT in terms of MOTA, MOTP, and IDF1. In particular, it significantly reduces false matches and improves tracking continuity, demonstrating the method’s effectiveness and reliability in complex underwater multi-target tracking scenarios.