Radiologists’ Responses to Reliable and Unreliable AI Confidence in Chest X-Ray and Ultrasound Interpretation: A Multi-Reader Study of Diagnostic Accuracy, Trust, and Automation Bias: A Systematic Review

Chest radiography is the most widely performed imaging examination worldwide and serves as a first-line test across emergency, inpatient and outpatient care. Its interpretation is nevertheless demanding, and diagnostic errors, including missed pneumonia, pulmonary nodules and pleural effusions, are frequent even among experienced readers [1]. Rising imaging volumes and radiologist workforce constraints have accelerated interest in artificial intelligence (AI) decision support, and by 2022 about 190 radiologic AI programs had received approval from the United States Food and Drug Administration, with approvals increasing each year [2,3]. Controlled reader studies show that AI assistance can improve performance when the algorithm is accurate. In a retrospective multireader multicase study, the macro-averaged area under the curve (AUC) of radiologists across 127 chest radiograph findings rose from 0.713 unaided to 0.808 with AI assistance [4]. Another reader study reported higher AUROCs for pneumothorax (0.885 to 0.969) and nodules (0.724 to 0.752), together with a roughly 10% reduction in reporting time [5]. These results have supported the view that AI will help readers by default. AI output is, however, never perfectly accurate. A review of 503 algorithm studies, cited in recent human-factors work in radiology, reported pooled sensitivity of 0.87 and specificity of 0.89 for chest radiograph abnormalities, implying that about 13 of 100 abnormal examinations and 11 of 100 normal examinations would receive an erroneous output [6]. The clinical value of AI therefore depends not only on algorithm accuracy but also on how readers respond when the algorithm is wrong.