Validation of an Automated ASPECTS Software via a Multi-Reader Multi-Case Clinical Reader Study

BACKGROUND: Rapid, accurate detection of early ischemic changes (EIC) on non-contrast computed tomography (NCCT) is critical for the triage and treatment of acute ischemic stroke (AIS) patients, yet NCCT interpretation remains challenging due to low soft-tissue contrast and inter-reader variability. MATERIALS AND METHODS: This study validates StrokeSENS ASPECTS, a fully automated AI-based software tool designed to aid clinicians in grading region-level EIC. A fully crossed multi-reader multi-case (MRMC) clinical reader study was conducted with eight clinicians who independently scored 100 NCCT scans from patients with confirmed middle cerebral artery (MCA)/internal carotid artery (ICA) occlusion, unaided and aided by StrokeSENS ASPECTS, with a three-expert-neuroradiologist consensus serving as the reference standard. Reader performance was assessed using binary classification metrics, including balanced accuracy, overall accuracy, sensitivity, and specificity; inter-reader agreement was assessed using Fleiss’s kappa. RESULTS: The use of StrokeSENS ASPECTS improved readers’ balanced accuracy by 5.7 percentage points, whereas overall accuracy, sensitivity, and specificity improved by 2.6, 9.7, and 1.6 percentage points, respectively, when compared to the unaided baseline (p < 0.001). Inter-reader agreement showed a significant increase in Fleiss’s Kappa of 0.285 from 0.323% (unaided) to 0.608% (aided). CONCLUSION: StrokeSENS ASPECTS have shown to improve clinicians’ ability to detect EIC on NCCT and reduce inter-reader variability. This demonstrates StrokeSENS ASPECTS’s safety and effectiveness as an aid in the evaluation of AIS.