Deep learning classification of reproductive tissue from ultrasound: sex determination in red abalone (Haliotis rufescens)

Introduction Accurate sex determination is critical for spawning success in both conservation breeding programs and commercial aquaculture, yet non-invasive methods remain limited in abalone species. Traditional approaches rely on visual inspection, which requires substrate detachment, can cause injury, and may induce premature gamete release. Here, we present the first application of machine learning to automate sex classification in red abalone (Haliotis rufescens) using non-invasive ultrasound imaging technology. Methods We developed a labeled dataset of 246 high-quality ultrasound images from 44 individuals and benchmarked seven convolutional neural network architectures: VGG16, VGG19, ResNet50, ResNet101, YOLOv8, YOLOv11, and a custom convolutional neural network. Data partitioning by individual identity was essential to prevent artificially inflated accuracy from image leakage across splits. Results The YOLOv8 architecture achieved the highest test accuracy of 85.7% (precision: 0.905 male, 0.816 female; recall: 0.845 male, 0.899 female), outperforming both classical architectures and custom models. Interestingly, a custom reverse VGG architecture with decreasing channel depth outperformed standard VGG models, suggesting that early channel compression may help combat ultrasound speckle noise. Discussion Feature activation maps confirmed that models learned to attend to gonadal tissue rather than imaging artifacts. We also demonstrate inference on NVIDIA Jetson edge devices, enabling real-time classification suitable for field deployment. This framework establishes the feasibility of automated, non-lethal sex determination for mollusks and lays the groundwork for applications to endangered abalone species where traditional invasive methods are prohibited.