Attention-driven Representation Enhancement and Uncertainty Refinement for Polyp Segmentation

Abstract Polyp segmentation is of great significance for the diagnosis and treatment of colorectal cancer. However, the accurate segmentation of polyps remains challenging due to the morphological and textural similarities between the lesions and surrounding tissues. In this paper, we propose an attention-based polyp segmentation method and target learning the visual patterns of lesion regions from the perspectives of representation enhancement and uncertain region refinement. To achieve this, we design Co-attentive Representation Learning (CaRL), which integrates spatial and channel attention mechanisms to effectively enhance the representation by emphasizing critical regions and features. We also propose the Attention-based Uncertainty Refinement (AUR) module, which utilizes attention mechanisms to gradually eliminate false negatives and false positives by refining uncertain regions. Deep networks implemented with CaRL and AUR enhance and refine discrepant yet discriminative features for polyp segmentation in an end-to-end manner. Experimental results demonstrate that our proposed method significantly improves the performance of polyp segmentation in both white light and narrow-band imaging (NBI) modalities while achieving state-of-the-art performance.