Automated Dentate Nucleus Segmentation from QSM Images Using Deep Learning

Abstract Purpose To develop a dentate nucleus (DN) segmentation tool using deep learning (DL) applied to brain quantitative susceptibility mapping (QSM) images. Materials and Methods Brain QSM images from 132 healthy controls and 170 individuals with cerebellar ataxia or multiple sclerosis were collected from nine different datasets worldwide for this retrospective study. Manual delineation of the DN (gray matter and white matter hilus) was first undertaken by experienced raters with a robust quality control process. Performance of automated segmentation was compared following training using several DL architectures. A two-step approach was implemented, composed of a localization model followed by DN segmentation. Results The manual tracing protocol produced ground-truth data with high intra-rater (average ICC 0.906) and inter-rater reliability (average ICC 0.776). Initial DL architecture exploration indicated that the nnU-Net framework performed best. The two-step localization plus segmentation pipeline achieved a Dice score of 0.898±0.031 and 0.894±0.036 for left and right DN, respectively. In external validation, our algorithm outperformed the leading existing automated tool (left/right DN Dice 0.863±0.038/0.843±0.066 vs. 0.568±0.222/0.582±0.239). The model demonstrated generalizability across unseen datasets during the training step. The measures showed a superior correlation index with manual annotations and performed well in both isotropic and anisotropic QSM datasets. Conclusion We provide a model that accurately and efficiently segments the DN from brain QSM images. The model can be readily deployed for use in observational, natural history, and treatment trials for biomarker discovery.