Demodulation Based Transformer for rPPG Generation and Heart Rate Estimation

As a convenient, efficient and inexpensive medical technology, heart rate (HR) estimation from face videos has gradually become a hot topic and many methods have been exploited to learn a spatial-temporal map for HR estimation. However, these methods have poor performance under time-varying ambient lighting. The main reason is that the current methods neglect the optical modeling of extracting the contactless physiological signal from skin. In this study, we describe the signal extraction as a modulation process and draw the conclusion that light changing could produce amplitude modulation jamming after modeling analysis. Therefore, a demodulation-based Transformer is newly designed for rPPG signal purification. In addition, a Pwelch-based Softmax operation is incorporated for HR estimation to improve accuracy. Finally, the hybrid loss combined with the negative Pearson correlation coefficient and cross-entropy loss is introduced for entire network learning. The experimental results on two databases (COHFACE and PURE) are performed to verify the effectiveness of the proposed method.