Answers are grounded in this paper and cite their sources.
Evaluates the impact of personalization and dataset characteristics on sleep classification performance.
Study Objectives Methods to improve sleep classification of wearable data often emphasize model choice. However, without widely used benchmark datasets, it is difficult to determine factors driving performance gains (model architecture, dataset selection, or evaluation decisions). This study examines how personalization, validation strategy, and dataset characteristics influence sleep classification performance beyond model choice. Methods We collected SleepAccel Clinical, a dataset of Apple Watch acceleration and polysomnography from 31 individuals with sleep apnea. Neural network models were trained on this dataset alongside SleepAccel (31 healthy individuals) and DREAMT (100 individuals with sleep disorders) to evaluate how training and testing set choices impact performance. All models were compared against an Easy To Classify (ETC) wake model, which estimates wake probability by smoothing and scaling activity in a brief time window. The ETC model’s average area under the recever operating curve (AUROC) on each dataset was used to quantify intrinsic ease of classification. Results The ETC model accounted for substantial amount of the models’ performance across the architectures, datasets, and training conditions, with significant correlations (p < .01) between ETC performance and all other scenarios. Including individuals with obstructive sleep apnea (OSA) in the training data improved performance when testing on datasets in which sleep disorders were suspected. Conclusions Sleep classification performance depends heavily on training and testing dataset characteristics, not solely on model choice. The presence of ETC wake epochs can markedly inflate performance metrics. Researchers should benchmark models on widely available datasets and quantify intrinsic class separability when introducing new datasets. This paper is part of the Consumer Sleep Technology Collection.
No takes yet. Share an insight, caveat, or question.
Cantón et al. (2026) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: