Comparative Assessment of CNN and Transformer U‐Nets in Multiple Sclerosis Lesion Segmentation

ABSTRACT Multiple sclerosis (MS) is a chronic autoimmune disease that causes lesions in the central nervous system. Accurate segmentation and quantification of these lesions are essential to monitor disease progression and evaluate treatments. Several architectures are used for such studies, the most popular being U‐Net‐based models. Therefore, this study compares CNN‐based and Transformer‐based U‐Net architectures for MS lesion segmentation. Six U‐Net architectures based on CNN and transformer, namely U‐Net, R2U‐Net, V‐Net, Attention U‐Net, TransUNet, and SwinUNet, were trained and evaluated on two MS datasets, ISBI2015 and MSSEG2016. T1‐w, T2‐w, and FLAIR sequences were jointly used to obtain more detailed features. A hybrid loss function, which involves the addition of focal Tversky and Dice losses, was exploited to improve the performance of models. This study was carried out in three steps. First, each model was trained separately and evaluated in each dataset. Second, each model was trained on the ISBI2015 dataset and evaluated on the MSSEG2016 dataset and vice versa. Finally, these two datasets were combined to increase the training samples and assessed on the ISBI2015 dataset. Accordingly, the R2U‐Net and the V‐Net models (CNN‐based) achieved the best ISBI scores among the other models. The R2U‐Net model achieved the best ISBI scores in the first and last steps with average scores of 92.82 and 92.91, while the V‐Net model achieved the best ISBI score in the second step with an average score of 91.28. Our results show that CNN‐based models surpass the Transformer‐based U‐Net models in most metrics for MS lesion segmentation.