Deep learning-based keypoint detection for 3D biometric measurement of tuna using underwater stereo vision

Accurate and scalable estimation of tuna size during transfer and farming operations is essential for stock assessment and management. This study presents a fully automatic processing pipeline that combines deep learning-based keypoint detection with stereoscopic vision geometry to estimate fish length, height, and width from underwater stereoscopic imagery. The system exploits videos acquired while freely swimming fish under real underwater conditions from two different viewpoints: a lateral view, from which length and height are measured, and a ventral view, which provides length and width. Having measurements of individual fish in all three dimensions can significantly improve the accuracy of estimating biomass and schools for large-bodied species. For each view, a Keypoint R-CNN model was trained on heterogeneous datasets of 850 stereo pairs annotated independently by three human operators. To account for annotation variability, per-keypoint normalization constants were computed and used to adapt COCO-style Object Keypoint Similarity (OKS) metrics to this domain. The proposed Keypoint R-CNN system achieved 2D keypoint localization accuracy comparable to human annotators (mean OKS = 0.91). The resulting 3D biometric measurements yielded relative errors of approximately 1%–2% for length-related dimensions and 2%–5% for height and width, within the range observed for inter-annotator variability. Measurement bias remained below 0.2% for all evaluated dimensions, and no systematic over- or under-estimation trends were observed. Automatic and manual size distributions were statistically indistinguishable, while processing throughput was compatible with offline operational analysis requirements. Overall, the proposed approach provides a reliable and scalable foundation for automated 3D size monitoring in tuna aquaculture.