Interpreting antiretroviral therapy loss to follow-up predictive models using LIME and SHAP

Loss to follow-up (LTFU) among people receiving antiretroviral therapy (ART) remains a challenge for sustained HIV treatment and retention in care. Early identification of patients who may be at increased risk of LTFU using machine learning techniques could support timely targeted retention interventions. This study evaluated the predictive performance and interpretability of Random Forest (RF) and Extreme Gradient Boosting (XGBoost) models for predicting LTFU among patients receiving ART. This study implemented and evaluated the predictive performance and interpretability of RF and XGBoost using Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP) for predicting LTFU among patients receiving ART. The study utilised data from Zimbabwe’s Ministry of Health and Child Care’s Electronic Patient Monitoring System, spanning May 2023 to April 2024. A dataset comprising 1,182 ART patients was analysed, of whom 294 (24.87%) experienced LTFU, and 888 (75.13%) were retained in care. SMOTENC was applied within the training folds to address class imbalance while avoiding resampling of validation data. The performance of models was evaluated using repeated stratified five-fold cross-validation with 10 repeats, resulting in 50 validation estimates. Performance was assessed using accuracy, precision, recall, F1-score, balanced accuracy, Matthews correlation coefficient, Cohen’s kappa, ROC AUC and PR AUC. Out-of-fold predictions were additionally assessed for calibration and threshold-dependent performance. SHAP was used to assess global feature contributions, and LIME was used to explain representative individual predictions. RF and XGBoost models show high predictive performance. Random Forest achieved a mean accuracy of 0.9751 ± 0.0093, recall of 0.9810 ± 0.0170, F1-score of 0.9517 ± 0.0175, and ROC AUC of 0.9844 ± 0.0095. XGBoost achieved corresponding values of 0.9726 ± 0.0091, 0.9677 ± 0.0239, 0.9462 ± 0.0176, and 0.9855 ± 0.0081. XGBoost produced marginally higher precision, ROC AUC and PR AUC, whereas RF showed higher performance across several threshold-dependent measures. The paired RF–XGBoost difference in ROC AUC was − 0.0011 (95% CI:−0.0026 to 0.0006), indicating that the observed difference was compatible with no difference. Brier scores were 0.0221 for RF and 0.0233 for XGBoost. SHAP analysis showed substantial agreement between models, with current viral suppression, viral suppression at 12 months and CD4 category among the most influential predictors. LIME shows how these variables contributed to individual predictions, while false-positive and false-negative cases showed that highly confident predictions could still be incorrect. Random Forest and XGBoost both showed high internal predictive performance for predicting LTFU among patients receiving ART, with broadly comparable discriminatory ability. The findings suggest potential for explainable machine learning techniques to support targeted ART retention strategies.