Greater value add from electronic health records than polygenic risk scores for predicting myocardial infarction in machine learning

Figures

• 

  Fig. 1. Integration of polygenic risk scores (PRSs) with clinical risk models across varying feature spaces and model complexities. LR logistic regression, NN feedforward neural network, NG1 non-genetic dataset 1 (known disease-specific risk factors), NG2 non-genetic dataset 2 (large-scale medical history embedding + known disease-specific risk factors), AUC area under the curve, PRS polygenic risk score. a The established risk factors dataset (NG1) comprised in-clinic measurements, lab test results, and self-reported questionnaire data on lifestyle and demographic factors associated with myocardial infarction risk. For the larger-scale clinical dataset, diagnostic data were extracted from five UK Biobank datasets, including electronic health records (EHRs), and these data were then preprocessed and used collectively in an autoencoder-based representation learning algorithm (see Methods for details) to generate feature embeddings containing data on each individual’s medical history. These diagnostic features were then combined with the NG1 dataset to form the NG2 non-genetic dataset. The two model complexity categories explored comprised logistic regression and neural network, whereby…

• 

  Fig. 2. Performance gains from integrating polygenic risk scores (PRSs) across clinical feature sets and model complexities. LR logistic regression, NN feedforward neural network, NG1 non-genetic dataset 1 (known disease-specific risk factors), NG2 non-genetic dataset 2 (large-scale medical history embedding + known disease-specific risk factors), AUC area under the curve, PRS polygenic risk score. a The performance, as measured by AUC, is shown for each of the eight models trained. Specifically, for each model complexity and feature space category, the respective NG and NG + PRS model performance is shown. The inclusion of PRSs provides a small but significant improvement in model performance regardless of the variables modified. The greatest magnitude improvement in performance over the baseline (NG1; LR) is achieved when large-scale diagnostic data are added to the model. b The value-add of the PRSs, as calculated with ΔAUCPRS ranging from 0.007 to 0.011 (ΔAUCPRS = AUCNG+PRS – AUCNG), is shown. The quadrants show the comparisons between NG and NG + PRS models for each category combination illustrated in Fig. 1b. While there is no significant improvement in PRS value-add over baseline…

• 

  Fig. 3. Feature importance and interpretability of risk contributions across data modalities. PRS polygenic risk score features, Abs absolute value, LR logistic regression, NN feedforward neural network, NG1 non-genetic dataset 1 (known disease-specific risk factors), NG2 non-genetic dataset 2 (large-scale medical history embedding + known disease-specific risk factors), SBP systolic blood pressure, BMI body mass index, Hx history, Sig. significant. Diagnoses refers to diagnostic features in the NG2 dataset, excluding NG1 features. a Boxplots with the distribution of feature importance weights for the different data modalities in NG2 + PRS; LR model. Medical history (i.e., diagnostic) features have significantly lower weights on average compared to established risk factors and PRSs. b A scatterplot comparing the absolute value of the mean logistic regression feature importance weight for each of the 589 features in the NG2 + PRS; LR model to the p value of each feature. The p values were calculated using a one-sample two-tailed t-test for each feature weight across ten trials against a null hypothesis of zero weight. Significant features were selected after correcting for multiple…

• 

  Fig. 4. Feature interpretation analysis for the most important diagnostic feature for incident myocardial infarction risk prediction. ICD-10 International Classification of Diseases, Tenth Revision, Pos. positive, Neg. negative, Char character. a Distribution of feature importance weights for 19,154 ICD-10 codes, split by hierarchy level in the coding scheme, from least granular (chapter-level) to most granular (5-character). The most relevant chapter-level and block-level codes have the largest absolute value importance weights, whereas the more granular codes individually contribute less towards prediction. b The 18,869 3- to 5-char codes (granular codes), which are more interpretable, are shown. The top 50 positive and negative codes have significantly larger magnitude feature importance than all others. c The top five positive and negative granular codes by mean feature importance for node 235. d Feature importance weights for 18,869 granular codes were aggregated. The top positive and negative code weights were then compared to other codes as a percentage of the total. The top 1000 positive codes and top 1000 negative codes combined (just ~11% of granular codes) comprise 91% of the…

• 

  Fig. 5. Feature interactions analysis for neural network models with small-scale clinical data (NG1 + PRS; NN and NG1; NN). BMI body mass index, SBP systolic blood pressure, BP blood pressure, Hx history, NG non-genetic features, PRS polygenic risk scores, NG1 non-genetic feature set 1, NN model complexity subcategory 2 (neural network). a A heatmap of all 171 pairwise interaction weights between the 19 features in the NG1 + PRS; NN neural network model. The features with the greatest number and magnitude of interactions included age, BMI, and male sex. The PRSs were amongst the least-interacting features, but CAD_6M was the score most enriched for feature interactions. The mean interaction weight across all ten trials is shown. b A histogram describing the distribution of mean pairwise interaction weights (across ten trials) for each of the NG1 + PRS; NN models and its counterpart neural network model with just non-genetic data (NG1; NN). The inclusion of PRSs skews the distribution towards having smaller numbers of mean interaction weights in general. Interactions between PRSs and other features contribute to ~39% of all 171 pairwise interactions on the NG1 + PRS model. c A bar chart…

• 

  Fig. 6. Feature importance and interactions for large-scale diagnostic features and PRSs in neural network models (NG2 + PRS; NN and NG2; NN). LR logistic regression, NN neural network, NG1 non-genetic dataset 1 (known disease-specific risk factors), NG2 non-genetic dataset 2 (large-scale medical history embedding + known disease-specific risk factors), PRS polygenic risk score, SBP systolic blood pressure, BMI body mass index, Hx history, Diag diagnostic features (NG2 feature set excluding the NG1 features). a (Left) Scatterplot of the mean feature importance weights across ten trials of the 589 NG2 features for the NG2; NN (neural network) model and its counterpart NG2; LR (logistic regression) model. The features are highly correlated (Pearson r2 = 0.96). (Right) The same is shown for the multi-modal NG2 + PRS; NN (neural network) model and its counterpart logistic regression model. b Bar chart of the number of significant features for the NG2; LR model and its counterpart neural network model (NG2; NN). Feature significance was calculated using a one-sample two-tailed t-test comparing the mean and standard error of each feature importance weight across ten trials to zero, followed by…