Multi-Task Learning with Graph-Guided Feature Fusion Module for Nuclear Instance Segmentation

Nuclear instance segmentation provides crucial nucleus morphology information for downstream computational pathology tasks, such as disease diagnosis and treatment prediction. In this regard, a typical class of solutions is proposal-based methods, which initially identify regions of interest (ROI) where individual instances are located and subsequently perform segmentation on these regions. However, nuclei in tissue images exhibit substantial variability across different organs, making it difficult for such approaches to accurately delineate boundaries and distinguish adjacent nuclei in dense scenes. To deal with the above issues, we propose a Global Semantic Segmentation (GSS) branch and a Graph-guided Feature Fusion (GFF) module in this work. Specifically, 1) the designed GSS branch utilizes a multi-scale convolution module to predict three-class semantic results for the entire image. This branch propagates features simultaneously with the detection branch. Such a design bridges the optimization gap between detection and segmentation tasks and enhances the network's ability to capture contextual information on the whole image. 2) While incorporating detection features and semantic features, we leverage the capabilities of graph convolution to discern the relationships between pixels. The GFF module produces pixel embeddings that estimate the similarity between each pair of pixels. Consequently, the network becomes more sensitive to differentiate between adjacent nuclear instances within a shared region. We conduct comparative experiments on the two datasets Lizard and PanNuke. The extensive experiments show the superiority of our proposed schemes.