Cross-sectional study evaluating the impact of an artificial intelligence-assisted system on diagnostic accuracy for chest and bone X-rays in the emergency department

This study aimed to evaluate whether two artificial intelligence (AI) systems for anomaly detection, ChestView for chest X-rays (CXR) and BoneView for bone X-rays (BXR), could improve the diagnostic accuracy of emergency physicians in a tertiary hospital emergency department (ED). We also compared the performance of radiologists in training and expert radiologists when assisted by AI with their performance in a previous evaluation conducted without access to these tools. We conducted an observational, cross-sectional diagnostic evaluation study in a tertiary hospital ED. Observers with access to the two AI systems with the ability to detect determined anomalies interpreted 347 CXR and 279 BXR that were selected prospectively and consecutively from ED patients. The reference diagnosis (“gold standard”) was established by consensus among radiologists, supported by clinical information and additional imaging when available. Diagnostic validity was assessed using sensitivity, specificity, positive predictive value, and negative predictive value (NPV), and observer performance was compared with results from a previous study of the same observers conducted without AI support. For CXR, AI assistance significantly improved the sensitivity of the emergency physician for overall anomaly detection (49.2% to 78.8%, p < 0.001) and increased NPV (76.3% to 94.8% p < 0.001). Improvement was also seen in detecting pleural effusion (41.4% to 77.8%, p = 0.002) and pulmonary consolidation (36.5% to 63.4%, p = 0.01). For BXRs, sensitivity improved from 77.6% to 86.6% and NPV from 93.5% to 95.5%, though differences were not statistically significant. However, a paradoxical drop in the sensitivity of emergency physicians to detect dislocations (80.0 to 18.2%, p = 0.018) was observed when they were assisted by AI. AI–assisted systems showed potential to enhance the performance of emergency physicians in detecting chest and bone abnormalities, particularly in pulmonary pathologies, by increasing sensitivity and NPV. These tools may help reduce diagnostic omission errors and optimize ED workflow. However, the observed decline in performance for dislocation detection highlights the need for caution. As AI adoption expands, balanced integration, independent evaluation, and awareness of limitations are essential to prevent overreliance and ensure safe, effective clinical application.