Clinical validation of autoSCORE: A human-in-the-loop AI tool for interpretation of long-term EEG recordings

This is the authors' abstract. We don't add key points for this paper.

Objective To validate autoSCORE, a convolutional neural network (SCORE-AI) implemented into a commercially available EEG reader, for analysis of long-term EEG (LT-EEG) recordings, and to evaluate it within the human-in-the-loop approach. Methods autoSCORE automatically divides LT-EEG into 2-hour segments, labeling each as normal or as containing one or more of four abnormality categories: focal epileptiform, generalized epileptiform, focal non-epileptiform, or diffuse non-epileptiform. In abnormal segments, it also inserts markers defined by start/stop times and tagged by abnormality type. Within each category, markers can be ranked by decreasing probability. Eighty 2-hour LT-EEG segments (40 in-hospital, 40 ambulatory) from patients aged 0.9–82.3 years were analyzed by autoSCORE. The diagnostic reference standard was derived from independent conventional visual assessment by three human experts. Results Accuracy computed across expert-expert and AI-expert pairings, evaluated at the level of the 2-hour segments, were comparable, except for detection of focal non-epilepiform abnormality, where AI outperformed experts for identification accuracy (p = 0.019). The positive predictive value of the automated abnormality markers was 0.446–0.823. In the human-in-the-loop workflow, inspecting 10 automated markers with the highest probability in each category enabled correct identification of all abnormal segments, corresponding to 1.5%–5.5% of the full 2-hour segment. Conclusions autoSCORE accurately screens LT-EEG recordings and may serve as an adjunct to human EEG interpretation to reduce the workload of human experts. Significance autoSCORE is a validated tool that can increase the productivity and efficacy of human experts in assessment of LT-EEG to improve patient diagnosis and management strategies.