A Semi-Convolutional Framework for Effective Instance Segmentation of Melanoma in Dermatological Images Using Spatially Enhanced Embedding Techniques

Abstract Melanoma segmentation has traditionally been addressed using region-based methods such as Mask RCNN, which are effective but often limited by their reliance on bounding box approximations. Recently, there has been a growing shift towards pixel-level segmentation tasks, as they offer the potential for greater precision and more seamless integration into end-to-end image-to-image frameworks. This paper explores the theoretical and empirical challenges of achieving instance-level melanoma segmentation through dense pixel embeddings, highlighting the limitations of conventional convolutional operators in isolating distinct instances. We propose semi-convolutional modifications, which enhance the capability of the model to distinguish between different instances of melanoma, by adapting Hough voting techniques and introducing a variant of the bilateral kernel that is spatially guided by a convolutional network. Our findings show that these novel operators not only improve the accuracy of melanoma segmentation but also outperform traditional region-based methods like Mask RCNN, especially in handling the complex and irregular shapes of melanomas, demonstrating significant improvements over previous techniques in both biological and medical imaging tasks.