Inter‐rater reliability on rhythmic and periodic EEG patterns using Standardized Computer‐based Organized Reporting of EEG ( SCORE ): A multicenter study

Objective This multicenter study aimed to evaluate inter‐rater reliability (IRR) for rhythmic and periodic patterns (RPPs), background activity, and interictal findings using the Standardized Computer‐based Organized Reporting of EEG (SCORE) across 10 European Reference Network (ERN) EpiCARE centers. Methods Fifteen neurologists and neurophysiologists, divided into two panels, independently analyzed a total of 277 adult electroencephalography (EEG) recordings, including RPP cases and controls without RPPs, using 5‐min EEG epochs. IRR was measured among pairs of raters and quantified using the maximum‐likelihood kappa (κML), an unbiased reliability estimator. EEG studies evaluated by at least six raters were included in the analyses ( n = 263). Results The overall identification of RPP's presence vs absence achieved substantial agreement (κML: .61–.66). Reliability for Main Terms 1 and 2 was substantial to almost perfect (κML: .73–.89), matching or exceeding previously published results. Notably, individual RPP types (κML up to .98) and complex modifiers, such as evolving dynamics (κML: .74–.87) and triphasic morphology (κML: .60–.88), achieved high reliability. Although background activity components showed almost perfect consensus (κML: .84–.98), agreement for posterior dominant rhythm was lower (κML: .06–.47). In contrast, the identification of sporadic interictal epileptiform discharges was high (κML: .89). Significance The SCORE framework enabled a detailed and comprehensive classification of RPPs, while also capturing background activities and other non‐RPP findings in a setting that closely resembles clinical practice. It highlighted complex characteristics for which IRR remains low.