Functional Outcome Prediction in Ischemic Stroke: A Comparison of Machine Learning Algorithms and Regression Models

Stroke-related functional risk scores are used to predict patients’ functional outcomes following a stroke event. We evaluate the predictive accuracy of machine-learning algorithms for predicting functional outcomes in acute ischemic stroke patients after endovascular treatment. Data were from the Precise and Rapid Assessment of Collaterals with Multi-phase CT Angiography (PROVE-IT), an observational study of 614 ischemic stroke patients. Regression and machine learning models, including random forest (RF), classification and regression tree (CART), C5.0 decision tree (DT), support vector machine (SVM), adaptive boost machine (ABM), least absolute shrinkage and selection operator (LASSO) logistic regression, and logistic regression models were used to train and predict the poor 90-day functional outcome, which is measured by the modified Rankin scale (mRS) score > 2. The models were internally validated using split-sample cross-validation and externally validated in the INTERRSeCT cohort study. The accuracy of these models was evaluated using the area under the receiver operating characteristic curve (AUC), Matthews Correlation Coefficient (MCC), and Brier score. Of the 614 patients included in the training data, 249 (40.5%) had poor 90-day functional impairment. The median and interquartile range (IQR) of age and baseline NIHSS scores were 77 years (IQR = 69 -83) and 17 (IQR = 11 - 22), respectively. The regression-based and machine learning models had comparable predictive accuracy when validated internally and externally (AUC range = 0.62 – 0.72; MCC range = 0.26 - 0.43). However, the former had better model calibration than the latter. Machine learning algorithms and logistic regression had comparable accuracy for predicting stroke-related functional outcomes in stroke patients. We recommend that the choice between among these classes of models should be guided by important considerations such as study design characteristics, type of data, data quality, and its utility in clinical settings.