What question did this study set out to answer?

To assess the diagnostic performance of commercial AI systems compared to radiologists in detecting pulmonary nodules using chest CT scans.

June 20, 2026Open Access

Diagnostic performance of commercial AI systems versus participating radiologists for pulmonary nodule detection in routine clinical practice

Key Points

To assess the diagnostic performance of commercial AI systems compared to radiologists in detecting pulmonary nodules using chest CT scans.
Retrospective analysis of chest CT scans for 102 patients, focusing on 353 nodules or masses.
Detection performance compared between two board-certified radiologists and two commercial AI systems against a reference standard.
Sensitivity, false positives per scan, and positive predictive value were evaluated along with logistic regression analysis.
Radiologists showed higher sensitivity (90.4-94.3%) compared to AI systems (80.2-89.2%).
Radiologists had significantly fewer false positives per scan (1.61-3.56) than AI systems (5.28-7.97; p < 0.001).
Limitations varied: radiologists struggled with intrinsic features while AI performance was hindered by case complexity and atypical nodules.

Abstract

PURPOSE: To evaluate the diagnostic performance of two commercial artificial intelligence (AI) systems versus that of radiologists on routine clinical chest computed tomography (CT) and to identify the imaging characteristics that limit AI performance. MATERIALS AND METHODS: We retrospectively analyzed the 5-mm-slice chest CT of 102 patients (353 nodules or masses). The detection performance of two board-certified radiologists and two commercial AI systems was compared against an expert-established reference standard. Sensitivity, false positives (FPs) per scan, and positive predictive value (PPV) were evaluated. Logistic regression identified factors influencing detection. RESULTS: The radiologists demonstrated higher sensitivity (90.4-94.3%) than the AI systems (80.2-89.2%) and significantly fewer FPs per scan (1.61-3.56 vs. 5.28-7.97; p < 0.001), resulting in superior PPVs. Multivariate analysis revealed divergent limitations: radiologists were challenged by intrinsic features (e.g., ground-glass nodules GGNs), whereas AI performance was degraded by case-level complexity (multiple nodules), specific locations (central), and atypical morphologies (large masses). CONCLUSION: In this study, the participating radiologists outperformed the commercial AI systems in routine clinical settings. Both exhibited distinct weakness profiles. Current AI systems are best suited as complementary tools rather than autonomous readers, provided FPs are managed effectively.

Perguntar à IA

Bookmark

View Full Paper