Addressing bias: artificial intelligence in cardiovascular medicine

Artificial intelligence (AI) is providing opportunities to transform cardiovascular medicine. As the leading cause of morbidity and mortality worldwide, cardiovascular disease is prevalent across all populations, with clear benefit to operationalise clinical and biomedical data to improve workflow, quality of patient care, and lower costs. AI technologies are designed to aid accurate decisions, without the bias of external factors—a pitfall of human errors. However, AI algorithms are still subject to their own biases. A particular challenge in cardiovascular medicine and the application of AI technology is in the intrinsic and extrinsic biases that affect quality control, and thus clinical outcomes. First, there is considerable heterogeneity in the quality of diagnostic studies and how they are done. For example, echocardiography is subject to both the operator doing the study and the interpreter analysing the study. Other cardiac imaging technologies, such as cardiac MRI, are not standardised in protocol, and vary by institution and machine vendor. These variabilities pose a challenge to the AI system, in which their learned ground truth is dependent on these human interpretations.1Lopez-Jiminez F Attia Z Arruda-Olson A et al.Artificial intelligence in cardiology: present and future.Mayo Clin Proc. 2020; 95: 1015-1039Summary Full Text Full Text PDF PubMed Scopus (20) Google Scholar Current studies using AI in cardiac imaging and diagnostics show great promise;2Rabbat M Leipsic J Bax J et al.Fractional flow reserve derived from coronary computed tomography angiography safely defers invasive coronary angiography in patients with stable coronary artery disease.J Clin Med. 2020; 9: 604Crossref Google Scholar, 3Stuckey TD Gammon RS Goswami R et al.Cardiac phase space tomography: a novel method of assessing coronary artery disease utilising machine learning.PLoS One. 2018; 13e0198603Crossref PubMed Scopus (10) Google Scholar, 4Oren O Gersh BJ Bhatt DL Artificial intelligence in medical imaging: switching from radiographic pathological data to clinically meaningful endpoints.Lancet Digit Health. 2020; 2: e486-e488Summary Full Text Full Text PDF PubMed Scopus (12) Google Scholar however, these studies might be limited to sample sizes at institutional or regionally distributed levels, or have not yet been externally validated by operators outside of the study or used on a separate population.1Lopez-Jiminez F Attia Z Arruda-Olson A et al.Artificial intelligence in cardiology: present and future.Mayo Clin Proc. 2020; 95: 1015-1039Summary Full Text Full Text PDF PubMed Scopus (20) Google Scholar Second, in machine learning models in which algorithms are designed to recognise patterns, there is a tendency to overfit the dataset because of an inability to distinguish a true contributing factor from noise.5Kagiyama N Shrestha S Farjo PD Sengupta PP Artificial intelligence: practical primer for clinical research in cardiovascular disease.J Am Heart Assoc. 2019; 8e012788Crossref PubMed Scopus (26) Google Scholar Thus, an overfitted model might show high performance in the training data that is then not applicable when shown a new dataset. In the training stage of the AI algorithm, it is thus essential to validate the algorithm in an independent dataset after the training is completed. Additionally, data processing requires regular annotation by trained clinicians to assist with quality control. The process is time consuming, and there is risk for the annotator to contribute their bias, either consciously or subconsciously, into the algorithm. Especially in AI-automated diagnostic procedures, there are often differing opinions among cardiologists. Annotations that might be perceived as significant by some, might be considered erroneous by others. The ideal process for annotating AI data would be through the consensus of multiple cardiologists; however, this would be an extremely expensive and time consuming approach. Another significant concern regarding the use of predictive AI models is the risk to worsen health disparities through biases in the data training the algorithm, and in the algorithm's intent. Women and minority groups are historically under-represented in cardiology, and the bulk of current evidence-based medicine might not necessarily apply to these populations. A systematic review of 207 trials found consistent under-reporting of female and Black patients from 2001 to 2018.6Tahhan AS Vaduganathan M Greene SJ et al.Enrollment of older patients, women, and racial/ethnic minority groups in contemporary acute coronary syndrome clinical trials: a systematic review.JAMA Cardiol. 2020; 5: 714-722Crossref PubMed Scopus (19) Google Scholar Although the same proportion of women and men present with chest pain, men are 2·5 times more likely to be referred to a cardiologist for management than women.7Clerc Liaudat C Vaucher P De Francesco T et al.Sex/gender bias in the management of chest pain in ambulatory care.Womens Health. 2018; 141745506518805641Google Scholar Black patients in the emergency room are 40% less likely to receive pain medication than White patients.8Lee P Le Saux M Siegel R et al.Racial and ethnic disparities in the management of acute pain in US emergency departments: Meta-analysis and systematic review.Am J Emerg Med. 2019; 37: 1770-1777Summary Full Text Full Text PDF PubMed Scopus (29) Google Scholar These inequalities are preserved in troves of health data, which are being used to train AI algorithms. Obermeyer and colleagues9Obermeyer Z Powers B Vogeli C Mullainathan S Dissecting racial bias in an algorithm used to manage the health of populations.Science. 2019; 366: 447-453Crossref PubMed Scopus (420) Google Scholar found that use of a widely used commercial prediction algorithm resulted in significant racial bias in predicting outcomes. Specifically, the algorithm identified White patients to have higher risk scores and were more often selected to receive additional care than Black patients who were equally as sick. In a study by Nordling and colleagues,10Nordling L A fairer way forward for AI in health care.Nature. 2019; 573: S103-S105Crossref PubMed Scopus (16) Google Scholar a machine learning algorithm identified the patient's postcode as the number one predictor for prolonged hospital stay, correlating to areas of low income and predominantly Black neighbourhoods. From these findings, another ethical dilemma arises in using an algorithm for decision making. If the algorithm was designed to optimise hospital resources, high-income White patients might be selected to receive the majority of hospital resources, further deepening the divide in access to care for minority and underserved groups. As AI technologies arise, it is crucial to monitor for unforeseen biases associated with socioeconomic and racial backgrounds of populations. The involvement of clinicians, ethicists, and representatives of minority groups early at the developmental stage of AI tools are integral to ensure that these predictive models represent the entire population. In the testing phase, AI algorithms should be validated on various minority and low-income groups to ensure that predictive models reflect the entire population. In the post-market stage, where algorithms get integrated into the health-care setting, regulatory and ethical guidelines are necessary to ensure that the benefits of receiving advanced AI technology are not limited to the affluent, but are equally distributed to all. Governing structures can implement a standard framework that would allow regular monitoring of AI models for quality and safety control. Rigorous models to validate results are necessary to minimise harm from use when it really matters—at the patient's bedside. DLB reports grants from Amarin, AstraZeneca, Bristol-Myers Squibb, Eisai, Ethicon, Medtronic, Sanofi Aventis, and The Medicines Company, Roche, Pfizer, Forest Laboratories/AstraZeneca, Ischemix, Amgen, Lilly, Chiesi, Ironwood, Abbott, Regeneron, Idorsia, Synaptic, Fractyl, Afimmune, and Lexicon; unfunded activities with FlowCo, Takeda, Medscape Cardiology, Regado Biosciences, Boston VA Research Institute, Clinical Cardiology, St Jude Medical (now Abbott), Biotronik, Boston Scientific, Merck, Svelte, Novo Nordisk, Cereno Scientific, and Cardiovascular Systems; grants and fees for participation on scientific advisory boards from PLx Pharma, Cardax, and PhaseBio; personal fees from Duke Clinical Research Institute, Mayo Clinic, Population Health Research Institute, Belvoir Publications, Slack Publications, WebMD, Elsevier, HMP Global, Harvard Clinical Research Institute (now Baim Institute for Clinical Research), Journal of the American College of Cardiology, Cleveland Clinic, Mount Sinai School of Medicine, TobeSoft, Bayer, Medtelligence/ReachMD, CSL Behring, Ferring Pharmaceuticals, MJH Life Sciences, Level Ex, Contego Medical, CellProthera, and K2P; personal fees and non-financial support from American College of Cardiology; personal fees and non-financial support from Society of Cardiovascular Patient Care; non-financial support from American Heart Association; and grants and personal fees from Boehringer Ingelheim. ET and MGR declared no competing interests.