Transformer based framework for nuclei segmentation and cell counting on microscopic images of multiple organs

Segmentation of cell nuclei in microscopic images is a vital task in cell counting, essential for cancer diagnosis and cancer grading. Also, it is more challenging due to the presence of overlapping nuclei which complicates the process of accurately distinguishing and separating individual nuclei. This study introduces an approach that combines Transformer-based encoder integrated with a U-Net architecture to improve the accuracy of nuclei segmentation. The proposed model was trained and validated on the 2018 Data Science Bowl dataset and demonstrated state-of-the-art performance, with a Dice-Similarity-Coefficient (DSC) of 92.6% and an Intersection-over-Union (IoU) of 86.22%. These metrics highlight the effectiveness of leveraging Transformers for capturing global contextual information, combined with the localized feature extraction capability of U-Net and enhancing segmentation accuracy. Additionally, a post-processing step was applied to refine the segmentation masks, improving boundary delineation and resolving overlapping nuclei. However, there exists several CNN models and deep learning models, this combination of advanced architectural design and post-processing techniques establishes a reliable framework for addressing the challenges of nuclei segmentation in biomedical imaging applications.