Memorizing Transformer for Small-scale Multi-parametric MRI Brain Tumor Diagnosis

Deep learning approaches have been widely applied to the MRI field. Among them, transformers have received increasing popularity due to their capability in handling multi modalities. Yet, transformers are hungry for large-scale data, which is expensive to collect. Here, we develop a novel memorizing transformer for small-scale multi-parametric MRI analysis. Empirically, we evaluate the proposed method on a dataset curated from 147 brain post-treatment malignant glioma cases for classifying treatment effect and tumor recurrence. The proposed memorizing transformer boosted a 5.15% improvement in the test area under the receiver-operating-characteristic curve (AUC) to the baseline transformer approaches.