Diagnostic Accuracy of On-Premise Automated Coronary CT Angiography Analysis Based on Coronary Artery Disease Reporting and Data System 2.0

Background Chest pain is a leading cause of outpatient and emergency department visits; advancements in artificial intelligence (AI) could improve coronary CT angiography (CCTA) workflows for these patients. Purpose To evaluate the performance of an on-premise AI-based coronary artery calcium scoring (CACS) and CCTA analysis software against expert interpretation based on Coronary Artery Disease Reporting and Data System (CAD-RADS) 2.0. Materials and Methods This retrospective study included consecutive patients undergoing CCTA for coronary analysis at a tertiary academic center between January 2017 and October 2021 across four scanners from three vendors. Patients with stents, bypass grafts, anomalies, or nondiagnostic studies were excluded. On-premise AI output included CACS, CAD-RADS category, and segment involvement score (SIS) within less than 5 minutes. Original CCTA reports were used as the reference, and discrepancies between AI and reports were further adjudicated by two blinded level-III readers with 8 and 5 years of CCTA experience. Agreement among CACS risk categories, CAD-RADS categories, and plaque burden scores was measured with the weighted κ. The area under the receiver operating characteristic curve, positive predictive value, and negative predictive value were used to evaluate diagnostic performance. Bootstrapping was used to estimate 95% CIs. Results A total of 1032 patients (median age, 62 [IQR, 54-69] years; 581 female) with 1041 CCTA images were included: 361 of the 1041 images (35%) were classified as CAD-RADS 0, 274 (26%) as CAD-RADS 1, 186 (18%) as CAD-RADS 2, 101 (10%) as CAD-RADS 3, 95 (9%) as CAD-RADS 4A, 11 (1%) as CAD-RADS 4B, and 13 (1%) as CAD-RADS 5. There was substantial agreement between AI and expert CAD-RADS stenosis severity categories (weighted κ = 0.73). AI demonstrated high performance (per-scan area under the receiver operating characteristic curve, 0.90; 95% CI: 0.87, 0.92) for CAD-RADS greater than or equal to 3 or greater than or equal to 4A and high negative predictive value (98%; 95% CI: 97, 99) but low positive predictive value (39%; 95% CI: 32, 45) for CAD-RADS greater than or equal to 4A. AI-based plaque burden scores derived from CACS reached near-perfect agreement with experts (weighted κ = 0.97), whereas those derived from SIS showed substantial agreement (weighted κ = 0.79). Conclusion On-premise AI accurately ruled out obstructive coronary artery disease at CCTA and achieved substantial to near-perfect agreement with human experts for CAD-RADS 2.0 stenosis severity and plaque burden.