A new statistical framework and test metric for evaluating seizure prediction algorithms against chance was derived and demonstrated in four patients undergoing intracranial EEG monitoring.
Does a new test metric comparing algorithm and chance sensitivities improve the statistical evaluation of seizure prediction algorithms in patients undergoing intracranial EEG monitoring?
The proposed statistical framework and test metric provide an advanced method for evaluating the performance of practical seizure advisory systems against chance.
Statistical methods for evaluating seizure prediction algorithms are controversial and a primary barrier to realizing clinical applications. Experts agree that these algorithms must, at a minimum, perform better than chance, but the proper method for comparing to chance is in debate. We derive a statistical framework for this comparison, the expected performance of a chance predictor according to a predefined scoring rule, which is in turn used as the control in a hypothesis test. We verify the expected performance of chance prediction using Monte Carlo simulations that generate random, simulated seizure warnings of variable duration. We propose a new test metric, the difference between algorithm and chance sensitivities given a constraint on proportion of time spent in warning, and use a simple spectral power-based measure to demonstrate the utility of the metric in four patients undergoing intracranial EEG monitoring during evaluation for epilepsy surgery. The methods are broadly applicable to other scoring rules. We present them as an advance in the statistical evaluation of a practical seizure advisory system.
Snyder et al. (Tue,) conducted a other in Epilepsy (n=4). New test metric (difference between algorithm and chance sensitivities) vs. Chance predictor was evaluated on Difference between algorithm and chance sensitivities given a constraint on proportion of time spent in warning. A new statistical framework and test metric for evaluating seizure prediction algorithms against chance was derived and demonstrated in four patients undergoing intracranial EEG monitoring.