Abstract No. 111 Analyzing Supervised Machine Learning Models for Prediction of Endoleaks Following Endovascular Aortic Aneurysm Repair

The purpose of this study was to evaluate the performance of various supervised machine learning algorithms to predict the occurrence of endoleaks following endovascular abdominal aortic aneurysm repair. This is an institutional review board–approved retrospective study that analyzed the demographic and clinical data of 72 patients with endoleak and 72 propensity-matched control patients with no endoleak. Seven machine learning models were selected: random forest, gradient boosting, decision tree, support vector machine, gaussian-naïve Bayes, multi-layer perceptron, and logistic regression. Sixty-four clinical and demographic variables were used as the feature matrix, which was standardized between a scale of 0 and 1 for both continuous and categorical variables. The data was split 70:30 amongst the training and testing sets, respectively. The training data was inputted into each machine learning model with their respective preset hyperparameters. Then, the testing data was analyzed by each trained model, and the predictions of each of the models were compared with the ground truth. The sensitivities, specificities, confusion matrices, and AUC curves were then calculated. Initial results demonstrated that all seven models performed with AUCs ranging from 0.39 to 0.51. A permutation feature importance (PFI) was performed on the 64 clinical data points that significantly improved the AUC of the models. Smoking, ASA, and COPD had the highest average feature importance for all models, with an average improvement of 0.123. After PFI was implemented, the Random Forest Classifier was the best-performing model, with a sensitivity of 48%, a specificity of 74%, and an AUC improvement from 0.39 to 0.65. The Support Vector Machine Classifier was the least accurate model, with a sensitivity of 52%, a specificity of 74%, and an AUC decrease from 0.51 to 0.35. Based on the clinical variables identified through the utilization of a PFI methodology, machine learning algorithms were able to further optimize predictive capacity in detecting endoleaks. Using PFI, we can identify the clinical data points that most significantly impact the performance of each model, leading us to optimize each model to identify the model that will best predict the occurrence of endoleaks after EVAR.