Convolutional Transformer-in-Transformer for Automatic Sleep Stage Classification

In this paper, a convolutional transformer-in-transformer architecture is proposed to capture local and global dependencies along time and frequency axes for automatic sleep staging. Existing sleep staging methods based on transformers have weaknesses in capturing local dependencies since they utilize fully-connected layers only. Inspired by the convolutional transformer in the computer vision domain that captures both local and global features well by taking advantage of both convolutional neural network and transformer, we propose to utilize convolutional projection with transformer-in-transformer architecture for feature extraction and aggregation. Numerical simulation results confirm that the proposed method outperforms existing sleep staging methods on the sleep heart health study (SHHS) dataset, in terms of macro F1-score.