Deep learning of brain lesion patterns and user-defined clinical and MRI features for predicting conversion to multiple sclerosis from clinically isolated syndrome

Multiple sclerosis (MS) is a neurological disease with an early course that is characterised by attacks of clinical worsening, separated by variable periods of remission. The ability to predict the risk of attacks in a given time frame can be used to identify patients who are likely to benefit from more proactive treatment. We aim to determine whether deep learning can extract latent MS lesion features that, when combined with user-defined radiological and clinical measurements, can predict conversion to MS (defined with criteria that include new T2 lesions, new T1 gadolinium enhancing lesions and/or new clinical relapse) in patients with early MS symptoms (clinically isolated syndrome), a prodromal stage of MS, more accurately than imaging biomarkers that have been used in clinical studies to evaluate overall disease state, such as lesion volume and brain volume. More specifically, we use convolutional neural networks to extract latent MS lesion patterns that are associated with conversion to definite MS (based on the McDonald 2005 criteria) using lesion masks segmented from baseline MR images. The main challenges are that lesion masks are generally sparse and the number of training samples is small relative to the dimensionality of the images. To cope with sparse voxel data, we propose utilising the Euclidean distance transform (EDT) for increasing information density by populating each voxel with a value indicating distance to the closest lesion boundary. To reduce the risk of overfitting resulting from high image dimensionality, we use a synergistic combination of downsampling, unsupervised pretraining and regularisation during training. A detailed analysis of the impact of EDT and unsupervised pretraining is presented. In contrast to our previous work, which only used automatically learned image features for prediction, we incorporate three user-defined magnetic resonance imaging (MRI) measurements and eight user-defined clinical measurements into the prediction model. In total, the baseline user-defined measurements consist of 11 features. Using the baseline MRI scans and all available measurements from 140 subjects in a sevenfold cross-validation procedure, we demonstrate that our model can achieve an average accuracy rate of 75.0% (SD = 11.3%) in predicting disease activity that is indicative of radiological or clinical conversion to definite MS within two years, which is higher than the 65.0% (SD = 14.6%) that is attained with lesion volume alone. More significantly, our model also outperformed a random forest using all available user-defined measurements (67.9%, SD=10.6%), thereby demonstrating the potential benefit of automatic extraction of latent lesion features by deep learning.