Unveiling diagnostic variability: clinical performance and pitfalls of two commercial AI tools for intracranial haemorrhage detection

Background and Objectives Artificial intelligence (AI) technologies have become increasingly prominent in emergency neuroimaging, yet most clinical studies have evaluated only single diagnostic models. Comparative evidence on the performance trade-offs between sensitivity and specificity in AI-based detection of intracranial haemorrhage (ICH) is limited. The objective of this study is to compare the diagnostic accuracy of two FDA-cleared AI models for detecting ICH on non-contrast brain computed tomography (NCBCT) and to explore the trade-offs between high sensitivity and high specificity algorithms. Methods In this single-centre, retrospective diagnostic accuracy study, consecutive adult patients undergoing NCBCT in the emergency department between 1 September 2024 and 28 February 2025 were included ( n = 3,884). Each CT was analysed by two FDA-cleared AI algorithms—NeuroSuite CT ICH Assessment System (EFAI-ICH) and Rapid ICH (RAPIDAI-ICH)—using 5 mm axial reconstructions. Model outputs were benchmarked against the final neuroradiologist report, with all false-positive and false-negative findings adjudicated. Performance metrics included sensitivity, specificity, accuracy, positive predictive value (PPV), and negative predictive value (NPV), with subgroup analysis by ICH subtype. Results Out of 3,884 CT, 421(10.8%) exhibited ICH, comprising 183 intraparenchymal, 56 intraventricular, 201 subarachnoid, 183 subdural, and 18 epidural subtypes. EFAI-ICH yielded 377 true-positives and 44 false-negatives (sensitivity 89.5%), whereas RAPIDAI-ICH yielded 308 true-positives and 113 false-negatives (sensitivity 73.2%). Specificity favoured RAPIDAI-ICH (97.1% vs 91.5%). Overall accuracy was higher for RAPIDAI-ICH compared to EFAI-ICH (94.5% vs 91.3%). EFAI-ICH demonstrated superior NPV (98.6%), while RAPIDAI-ICH achieved superior PPV (75.7%). Discussion Both AI platforms demonstrated robust diagnostic capabilities for emergency ICH detection, with complementary strengths. EFAI-ICH minimizes missed bleeds but at the cost of more false alarms; RAPIDAI-ICH excels at ruling out haemorrhage but risks under-calling subtle bleeds. Recognizing these performance variations is crucial for optimal AI integration into neuroradiological practice and ultimately enhancing patient care outcomes.