Coordinate-Aware Mask R-CNN with Group Normalization: A underwater marine animal instance segmentation framework

Unsustainable fishing, driven by bycatch and discards, harms marine ecosystems. Addressing this, we propose a Coordinate-Aware Mask R-CNN (CAM-RCNN) method to enhance fish detection in commercial trawls. Leveraging CoordConv and Group Normalization, our approach improves generalization and stability. To tackle class imbalance, a compound Dice and cross-entropy loss is employed, and image data are enhanced through multi-scale retinex and color restoration. Evaluating on two fishing datasets, CAM-RCNN excels in accuracy and generalization, achieving the best Average Precision (AP) for instance mask and BBOX prediction in both source (39.7%, 40.2%) and target domains (24.4%, 24.2%). This method promotes sustainable fishing by selectively capturing desired fish, reducing harm to non-target species.