Realistic Performance Estimation in Small-Cohort Deep Learning and Machine Learning: A Leakage-Controlled Codebase Applied to Meningioma Response Prediction After Gamma Knife Radiosurgery

Background/Objectives: Deep learning may support the prediction of meningioma response after Gamma Knife radiosurgery (GKRS), but small clinical cohorts increase the risk of overfitting and unstable performance. We developed leakage-controlled code adjusted for small clinical datasets. Methods: Pretreatment contrast-enhanced T1 MRI from 190 patients was analysed using one tumour-centred axial slice per patient. Response was defined as a tumour volume decrease ≥ 30% per log10 (months). ResNet-10t, ResNet-18, ResNet-50 and DenseNet-121 with ImageNet or RadImageNet pretraining were evaluated in three workflows: (1) frozen backbone CNN, head-only training; (2) logistic regression or naive Bayes classification of features extracted from frozen CNNs; and (3) restricted CNN fine-tuning. Five partially overlapping chronological development/test splits were assessed, with all data-dependent fitting, feature selection and model optimisation restricted to development data. Results: The smallest RadImageNet ResNet-10t CNN performed best when fine-tuned (peak AUCs 0.953 and 1.000) but showed the widest split-to-split variation (range 0.738–1.000). When RadImageNet features extracted from ResNet-10t were classified by naive Bayes and logistic regression, they reached lower median and peak AUCs but within narrower ranges. Predictive performance did not consistently depend on CNN size. Conclusions: To our knowledge, this is the first deep learning study predicting volumetric meningioma response after GKRS. We provide openly available, ready-to-use code for three major workflows in small clinical datasets, with safeguards against information leakage. The benefit of fine-tuning and of RadImageNet pretraining depended on architecture. Robust small-cohort modelling requires attention to performance consistency across chronological splits, not only to the highest AUC in a single split.