An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection

This is the authors' abstract. We don't add key points for this paper.

Accurate detection of rapid eye movement (REM) sleep is critical for neurophysiological monitoring and early identification of sleep-related disorders. An entropy-driven adaptive knowledge distillation framework integrated with a lightweight Transformer was developed to support robust REM sleep detection from single-channel electroencephalogram (EEG) signals. By dynamically adjusting temperature scaling and distillation weighting based on teacher–student entropy, the proposed approach explicitly models uncertainty inherent in REM sleep. When evaluated under subject-independent cross-validation on three public datasets (Sleep-EDF, ISRUC-Sleep, and MASS), the proposed model outperformed CNN, LSTM, and standard Transformer baselines, achieving an average accuracy of 93.2% and an F1 score of 92.8%. Moreover, the distilled student model reduced training time and memory usage by approximately 30%, indicating potential suitability for future optimization toward real-time and wearable EEG applications. These results suggest that uncertainty-aware adaptive distillation may support accurate and computationally efficient REM sleep detection from single-channel EEG under subject-independent experimental conditions.