Why the study?
Most existing deep learning remote heart rate estimation methods rely on convolutional neural networks that excel primarily at local feature extraction.
The proposed CMRPPGFormer network provides highly accurate remote heart rate estimation from facial videos, outperforming existing baseline methods.
Answers are grounded in this paper and cite their sources.
Vision transformer method may advance contactless HR monitoring; leaves open prospective clinical validation before practice adoption.
Most of the existing remote heart rate (HR) estimation methods based on deep learning networks use convolutional neural networks (CNNs) which excel at local feature extraction. This article proposes an HR estimation method based on a vision transformer network named as CMRPPGFormer. The basic idea of the proposed method is to extract precise remote photoplethysmography (rPPG) signals from facial videos by the CMRPPGFormer and then calculate relevant indicators of HR by specific postprocessing on rPPG signals. The CMRPPGFormer consists of the convolutional stem block, the feature fusion block, and the rPPG predictor block. The feature fusion block, the key of the CMRPPGFormer, can effectively extract and fuse local and global features of rPPG signals, in which a 3-D spatio-temporal convolutional modulation transformer (3D-ST-CMTransformer) model is proposed. The 3D-ST-CMTransformer extracts global features of rPPG signals by its 3-D convolution modulation attention layer and feedforward network (FFN) effectively. Experimental results on two public datasets, PURE and UBFC-rPPG, validate the better performance of the proposed method compared with the state-of-the-art baseline methods. Particularly, CMRPPGFormer obtains 0.39 beats per minute (bpm) MAE, 0.64 bpm RMSE, 0.99 r value of Pearson’s correlation coefficient and 0.59 bpm MAE, 1.77 bpm RMSE, 0.99 r value of Pearson’s correlation coefficient on the PURE and UBFC-rPPG datasets, respectively. Compared with the baseline methods, it achieves the lowest MAE and RMSE and the highest r.
No takes yet. Share an insight, caveat, or question.
Ma et al. (2024) studied this question.
Synapse has enriched 10 related papers. Consider them for comparative context: