An adaptive, variable‐length test for certifying electroencephalographic spike‐detection skill: A retrospective feasibility study

Objective Determining whether an individual reader has sufficient electroencephalographic (EEG) spike‐detection skill is challenging due to limited expert agreement. We evaluated whether modeling skill in interpretable signal‐detection terms—discrimination ability separate from response tendency—could power an adaptive, variable‐length test that can reliably and efficiently assess reader competence. Methods We analyzed a multirater dataset containing 20 521 EEG segments and 2574 rater sessions. Case signal was estimated from multirater labels, and each reader was modeled using signal‐detection parameters: discrimination skill and response bias . Among 368 calibration‐eligible raters across five skill tiers, we used a stratified 50/50 train–validation split to select a certification threshold in training and to apply it in validation. We replayed an adaptive test on each rater's historical responses, using sequential stopping to issue pass, fail, or retest decisions. Results The training‐derived threshold was (95% bootstrap confidence interval = –, Youden ). In validation, pass rates preserved the expected tier ordering: experts 100% (), experienced 100% (), novices 14% (), residents 0% (), and crowd raters 7% (). In the 81‐rater adaptive replay, clear failures stopped near 20 trials, and passing experts required a median of 95 trials. Split‐half replay showed intraclass correlation coefficient for at stopping; there were no hard pass–fail reversals among raters whose two halves both reached decisive verdicts. Significance A signal‐detection framework quantified spike‐identification skill in EEG readers, and adaptive testing estimated this skill reliably with fewer than 100 short EEG segments. This approach could provide a practical, scalable method for assessing EEG reading competence.