Machine learning and myocardial infarction diagnosis: sometimes you can’t make it on your own

Comment on ‘Machine learning for diagnosis of myocardial infarction using cardiac troponin concentrations’ which was published in Nature Medicine, DOI: 10.1038/s41591-023-02325-4. This study represents a prespecified analysis of the prospective, randomized, High-Sensitivity Troponin in the Evaluation of Patients with Suspected Acute Coronary Syndrome (High-STEACS) trial,1 aiming to develop a machine learning (ML)–based clinical decision support system that calculates the probability of myocardial infarction (MI) based on cardiac troponin (Tn) levels and clinical features, and to validate it externally in a cohort of patients with possible MI.2 The derivation cohort consisted of 10 038 patients (median age, 70 years; 48% female) with possible MI from the High-STEACS trial, enrolled at 10 secondary or tertiary care hospitals in Scotland between June 2013 and March 2016. The external validation cohort consisted of 10 286 patients (median age, 60 years; 35% female) with possible MI, enrolled in seven prospective cohort studies across six countries. Multiple ML models were trained using the derivation cohort to identify patients with a diagnosis of MI during the index admission, according to the Fourth Universal Definition of MI (including type I, type 4b, and type 4c), as adjudicated by two independent clinicians. Patients with ST-segment elevation MI were excluded. Single or serial high-sensitivity Tn I concentrations (measured using the ARCHITECTSTAT assay) and 12 clinical variables [i.e. age, sex, the number of hours from symptom onset to Tn I measurement, chest pain, known ischaemic heart disease, hyperlipidaemia, heart rate, systolic blood pressure, Killip class, evidence of ischaemia on electrocardiogram (ECG), estimated glomerular filtration rate, and haemoglobin] were used to compute the Collaboration for the Diagnosis and Evaluation of Acute Coronary Syndrome (CoDE-ACS) score (from 0 to 100). In patients without myocardial injury at presentation, defined as a cardiac Tn I concentration below the sex-specific 99th percentile upper reference limit on the first measurement, a CoDE-ACS score below 3 identified those at low probability of MI with a negative predictive value (NPV) of 99.5 [95% confidence interval (CI), 99.3–99.8]. In patients with myocardial injury at presentation, a CoDE-ACS score > 60 identified those at high probability of MI with a positive predictive value (PPV) of 80.1 (95% CI, 78.5–81.6). These scores performed consistently across subgroups, as well as after incorporating serial Tn I levels in the models. In the external validation cohort, the CoDE-ACS score showed excellent discrimination for MI, either using single Tn I measurements [area under the curve (AUC), 0.95; 95% CI, 0.95–0.96] or serial Tn I testing (AUC, 0.97; 95% CI, 0.96–0.97). As compared with fixed cardiac Tn I thresholds, a CoDE-ACS score below 3 identified more patients at low probability of MI (61% vs. 27%) with a similar NPV of 99.6%, while a CoDE-ACS score > 60 identified fewer patients at high probability (10% vs. 16%) with a greater PPV (75.5% vs. 49.4%). Based on the CoDE-ACS scores, patients at low probability of MI had lower rates of cardiac death compared to those at intermediate or high probability (0.3% vs. 2.8% vs. 4.2%, respectively, at 1-year follow-up; P < .001). Despite extensive research over the past decades, the rapid and accurate diagnosis of myocardial infarction (MI) in patients admitted to the emergency department remains a challenge.3 High-sensitivity troponin (Tn) assays are recommended by international clinical guidelines for the diagnosis of MI.4,5 Yet, these assays do not account for age, sex, or comorbidities, and, most importantly, may not identify Tn increases if sampling is carried out within a few hours of symptom onset.6 For this reason, current guidelines recommend serial sampling in early presenters, which adds complexity to patient’s management, often delaying diagnosis and increasing length of hospital stay and healthcare costs.4 Research efforts to develop machine learning (ML)-based decision support tools have been increasing in recent years. However, most studies enrolled highly selected patient populations and lacked external validation, did not use multiple parameters to train ML models, and often relied on serial Tn measurements to estimate the probability of MI.7–9 The current study by Doudesis and collaborators has the major strength of involving a large unselected patient population across multiple health care systems, estimating the probability of having MI using a single cardiac Tn measurement and including multiple clinical features that influence Tn concentration itself and pre-test probability of having MI.2 The use of the Collaboration for the Diagnosis and Evaluation of Acute Coronary Syndrome (CoDE-ACS) score resulted in a more precise identification of patients at low or high probability of MI than the use of isolated fixed cardiac Tn thresholds. By incorporating time from symptom onset, the CoDE-ACS score may allow early presenters to be ruled out using a single test. Indeed, positive predictive value for patients with high probability of MI based on the CoDE-ACS score was significantly higher than that obtained using fixed Tn I thresholds only, suggesting that CoDE-ACS may accelerate the diagnosis of MI by avoiding serial Tn assessments in early presenters. On the other hand, CoDE-ACS score identified twice as many patients at low probability of MI (absolute difference, 34%) with comparable negative predictive value as compared with fixed Tn thresholds. The potential impact of these findings on resource utilization and health care expenses would be substantial, if we consider that ∼20 million patients are admitted with suspected ACS to the emergency department in Europe and North America each year.4,10 In addition, patients at low probability of MI had a better clinical outcome up to 1 year, suggesting that CoDE-ACS may also aid clinical risk prediction. However, implementation trials are needed to prospectively evaluate how and when the decision support tool is used by clinicians, as well as its real-world impact on both performance in the diagnosis of MI and clinical outcome. This study has several limitations. Firstly, the strategy used to assess whether the ML models were accurate in identifying MI was based on a judgement made by the attending clinicians, and CoDE-ACS incorporated features that are integral to the diagnosis of MI, such as chest pain and evidence of ischaemia on electrocardiogram. Secondly, the study results are applicable exclusively to the Tn assay tested, and further data are needed to determine whether the same model might be applied to different Tn assays. Finally, the majority of patients were White, thereby limiting the generalizability of the results. In conclusion, this study represents an important step forward on the way to improve the identification and predict the clinical outcome of MI patients using ML. If the feasibility, safety, and efficacy of the CoDE-ACS score will be confirmed in future prospective studies, its adoption in the real-world practice might prove beneficial to both patients and healthcare systems. Supplementary data are not available at European Heart Journal online. R.V. received consulting or lecturing fees from Abbott Vascular, Amgen, Daiichi Sankyo, Medtronic, and Terumo, outside the submitted work. C.P. received consultant and speaker fees from AbbVie, Acticor Biotech, Amgen, Bayer, Eli Lilly, and Tremeau, and grant support (to the Institution) for investigator-initiated research from AIFA (Italian Drug Agency), Bayer, Cancer Research UK and European Commission; he chaired the Scientific Advisory Board of the International Aspirin Foundation.