SCB-Net: Self-supervised nuclei segmentation network with saliency guidance and contextual boundary attention

Accurate segmentation of nuclei is essential for structured analysis in digital pathology, directly influencing cancer diagnosis, grading, and prognosis. However, obtaining high-quality pixel-level annotations remains labor-intensive and time-consuming. Existing self-supervised learning methods primarily rely on low-level appearance representations to construct proxy tasks, which leads to limited sensitivity to nuclei morphology, suboptimal boundary discrimination, and insufficient modeling of spatial continuity. To mitigate these problems, a Self-Supervised Nuclei Segmentation Network with Saliency Guidance and Contextual Boundary Attention (SCB-Net) is proposed for instance segmentation of H&E stained images. First, SCB-Net incorporates the nuclei saliency guided self-supervision mechanism to generate structure-aware pseudo labels, which leverages cytological priors as structural driving signals to extract salient features by enhancing edge gradients and the structural continuity of nuclei. Then, during the segmentation stage, to enhance the structural expressiveness of the model in complex nuclei clusters and ambiguous boundary regions, the Contextual Boundary Attention (CBA) mechanism is designed to strengthen boundary directionality, reinforce edge gradients, and optimize cross-channel information fusion. Furthermore, the Efficient Triple Fusion (ETF) module is employed to concurrently fuse three complementary feature types (semantic, geometric structural, and visual boundary) to achieve cross-scale semantic alignment and geometrically consistent representations. Evaluations on the MoNuSeg, CPM17, and CryoNuSeg datasets demonstrate that SCB-Net attains AJI scores of 0.5609, 0.5102, and 0.4740, respectively. These results are comparable to weakly supervised methods despite requiring no ground-truth labels, and significantly surpass existing self-supervised approaches.