Adaptive physiology-informed correction for reliable remote photoplethysmography heart-rate monitoring

Figures

• 

  Fig. 1. The overall workflow of remote heart rate (HR) measurement in this study. This workflow is comprised of five parts: video recording; light intensity detection; red, green, and blue color channels (RGB) to remote photoplethysmography (rPPG) signal conversion; HR estimation; and HR correction. The facial ROI refers to the region of interest for skin pixel intensity extraction. Four rPPG algorithms are included in this research: plane-orthogonal-to-skin (POS)32, orthogonal matrix image transformation (OMIT)62, the chrominance-based method (CHROM)30, and local group invariance (LGI)45. Icons were adapted from the Microsoft 365 non-copyrighted icon library.

• 

  Fig. 2. Overall performance comparison on different measurement scenarios. a LGI-PPGI dataset45. b BUAA-MIHR dataset46. c UBFC-rPPG dataset47. The evaluation metrics of mean absolute error (MAE) and dynamic time warping (DTW) measured by beats per minute (BPM) were applied to evaluate the performance of our method and other methods for comparison. The notches of the box plots correspond to the median values. Statistical significance was obtained using bootstrap tests. Significance levels are indicated as (*p < 0.05), (**p < 0.01), (***p < 0.001), and n.s. (not significant).

• 

  Fig. 3. Bland–Altman analysis and the corresponding normalized frequency density of the HR estimation performance before and after being processed by our proposed correction algorithm. a LGI-PPGI dataset45. b BUAA-MIHR dataset46. c UBFC-rPPG dataset47. The Bland–Altman plots on the left side are conducted between the estimated HR versus the ground truth (GT). Each data point in the plots represents a pair of measurements within 25 seconds. The normalized frequency density on the right side is mapped from the data in the Bland–Altman plots. The acceptable range of HR measurement accuracy is set as ± 10 BPM according to the Consumer Technology Association (CTA) standards49,50.

• 

  Fig. 4. Representative examples of HR correction results produced by the proposed algorithm across different types of subject movements in the LGI-PPGI dataset45. a Exercising. b Rotation. c Talk. The estimated HR before and after correction, as well as the ground truth HR, are shown as blue, red, and black curves, respectively. The yellow polygon on the human face represents the glabella as the ROI. Above each facial video sequence, the subject code and MAE of HR estimation—averaged over the plane-orthogonal-to-skin (POS)32, orthogonal matrix image transformation (OMIT)62, chrominance-based (CHROM)30, and local group invariance (LGI)45 algorithms–are presented. The video frames of the LGI-PPGI dataset are under the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/).

• 

  Fig. 5. Representative examples of HR correction results produced by the proposed algorithm across different illumination levels in the BUAA-MIHR dataset46. a 6.3 lux. b 10.0 lux. c 15.8 lux. d 25.1 lux. The estimated HR before and after correction, as well as the ground truth HR, are shown as blue, red, and black curves, respectively. The yellow polygon on the human face represents the glabella as the ROI. Above each facial video sequence, the subject code and MAE of HR estimation–averaged over the plane-orthogonal-to-skin (POS)32, orthogonal matrix image transformation (OMIT)62, chrominance-based (CHROM)30, and local group invariance (LGI)45 algorithms–are presented. The video frames of the BUAA-MIHR dataset are under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License (https://creativecommons.org/licenses/by-nc-sa/4.0/).

• 

  Fig. 6. Working principle of the proposed adaptive HR correction algorithm. The facial video sequence is sampled from the LGI-PPGI dataset45, which is under the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/). The yellow polygon on the human face represents the glabella as the ROI. The power spectral density (PSD) of each time window is shown to further illustrate the HR estimation and correction process. HRI denotes the integer index mapped from HR. Representative scenarios I, II, III, and IV were presented using illustrative examples to further demonstrate the operation logic of the proposed HR correction algorithm. For the algorithmic notations, ti denotes the time of the i-th estimated HR, and HRIt and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${HRI}_{t}^{{\prime} }$$\end{document}HRIt′ denote the current HRI at time t before and after correction. Since the time interval between two adjacent HR estimations was set as 1 s, HRIt−1 and…