Adaptive Desaturation through Noise Injection: A Novel Approach to Enhancing Melanoma Detection

Abstract In recent years, convolutional neural networks (CNNs) have achieved remarkable success in various applications, including melanoma recognition. However, the training dynamics of CNNs can be hindered by certain optimization challenges. One such issue arises from the early saturation behavior in the softmax activation function, which can lead to suboptimal convergence when using stochastic gradient descent (SGD). This phenomenon may prevent the model from fully exploring the parameter space, often causing it to converge to poor local minima. To address this, we propose a novel technique called Noisy Softmax, which aims to alleviate the early saturation problem by introducing controlled noise into the softmax function during each training iteration. The noise, which is progressively annealed over time, helps to delay the saturation effect and enables more effective gradient propagation, thereby enhancing the model's ability to explore diverse solutions. Our approach fosters better convergence and improves the generalization capacity of CNNs in melanoma detection tasks. Extensive experiments demonstrate the effectiveness of our method, yielding competitive results compared to existing state-of-the-art techniques on several benchmark datasets for melanoma classification.