Real-time realizable mobile imaging photoplethysmography

Figures

• 

  Figure 1. Overview of the proposed system for HR estimation using a PPGI mounted on a robot; (a) Robotic device based on a turtlebot2 framework with a 3D camera, a laptop and a webcam; (b) SLAM and navigation; (c) Real-time HR estimation via face detection, face skin image extraction, PPGI acquisition; (d) Algorithm flow chart for SLAM and navigation.

• 

  Figure 2. Overview of the data flow for HR estimation.

• 

  Figure 3. Overview of face skin extraction (from the public dataset UBFC-rPPG): (a) detected face is outlined by a yellow rectangle and may include hairs and parts of the background; (b) histogram of S values, distributed around the center value \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$hist_{max}$$\end{document}histmax (the most frequent S value on the face image); (c) resultant face skin image after applying the relative saturation (S) value range.

• 

  Figure 4. Landmark-based face skin extraction: (a) a rectangular face ROI17, (b) a bottom face ROI33, and (c) a polygonal face ROI34.

• 

  Figure 5. Landmark-based face skin extraction: (a) a rectangular face region of interest (ROI)17, (b) a bottom face ROI33, and (c) a polygonal face ROI34.

• 

  Figure 6. (a) Pearson correlations between estimated HRs and true HRs (r = 0.9925). (b) Bland–Altman plot (μ = − 0.0315 and σ = 2.1005).

• 

  Figure 7. Comparison of the estimated HR results using our proposed method with and without the FSM framework. The HR is estimated using the proposed method, with detected noise removed by the FSM framework: (a) under the rapid head movement condition, (b) under the changes in ambient lighting condition.

• 

  Figure 8. Simultaneous HR estimation of two subjects (additional dataset): An ID is automatically assigned to each face, starting from one, and the face frames from each subject are converted to HR values (both participants provided written informed consent for the publication of identifying images in an online open-access publication).