Machine learning survival models (Kernal SVM C-index 0.7896) performed marginally worse than a Cox proportional hazards model (C-index 0.8082) for predicting 10-year CVD risk.
Cohort (n=1,776,865)
Yes
Do machine learning survival models improve 10-year cardiovascular risk prediction compared to a Cox proportional hazards model when incorporating social and environmental determinants in a primary care population?
Traditional Cox proportional hazards models performed marginally better than machine learning approaches for predicting 10-year cardiovascular risk when incorporating social and environmental determinants in a large primary care cohort.
Abstract Background and purpose Cardiovascular disease (CVD) risk prediction tools (i.e. QRISK) are widely embedded in UK primary care and underpin decisions identifying high risk individuals and initiating preventive interventions. However, these models rely on conventional clinical and demographic predictors and do not account for non-traditional determinants of risk. At the same time, most established tools are derived using traditional regression-based methods, despite growing evidence that machine learning (ML) approaches can enhance risk prediction performance in complex settings. This study aimed to integrate social determinants (area-level deprivation and air pollution) into a QRISK-based model and compare the results with ML survival models using a large, real-world primary care population. Methods We used longitudinal data from the Secure Anonymised Information Linkage (SAIL) Databank, including adults aged 25–84 years registered with general practices across Wales between 01/01/2001 and 31/12/2023. Incident CVD events were ascertained using linked primary care, hospital, and mortality records. In addition to standard QRISK predictors, we incorporated social and environmental determinants: area-level deprivation (Index of Multiple Deprivation IMD) and residential air pollution (mean annual NO2 concentrations) linked to patient addresses. We first developed a Cox proportional hazards model to estimate 10-year CVD risk, then compared its performance with several machine learning survival models (Random Forest, Support Vector Machine SVM, Gradient Boosting, Penalised Cox Regression, Neural Network). Model performance was evaluated using the concordance index (C-index), Brier score, area under the curve (AUC) at 10-years. Results The full cohort comprises of 1,776,865 individuals (50.6% female; mean age: 47.4 years SD:15.3), with a median follow-up time of 15.2 years (IQR: 5.8-22.9). During follow-up 254,040 (14.3%) first-time CVD events were recorded. Using 1% random sample of our cohort, we found the cox proportional hazards model performed best, yielding a C-index of 0.8082 (Table 1). This was followed by the Kernal Support Vector Machine (C-index: 0.7896). When performance was assessed using the Brier Score, Kernal SVM (0.0706), LASSO penalised regression (0.0709) and Elastic net penalised regression (0.0710) all performed well. The AUC at 10 years was best for the cox proportional hazards model (0.8528), followed by Kernal SVM (0.8285), LASSO penalised regression (0.8262). Conclusion Overall, the cox proportional hazards model performed marginally better compared to the ML models. Having initially used a 1% sample, our results may be biased. We aim to calculate confidence intervals for the performance measures, work towards model development using the full cohort, introduce further neighbourhood determinants (i.e. PM2.5 concentration) and compare performance measures with the UK clinical standard QRISK3.For image description, please refer to the figure legend and surrounding text. Figure 1:Proposed workflowFor image description, please refer to the figure legend and surrounding text.
Brown et al. (Mon,) conducted a cohort in Cardiovascular disease (n=1,776,865). Machine learning survival models vs. Cox proportional hazards model was evaluated on 10-year CVD risk prediction performance (C-index). Machine learning survival models (Kernal SVM C-index 0.7896) performed marginally worse than a Cox proportional hazards model (C-index 0.8082) for predicting 10-year CVD risk.