pyVHR: a Python framework for remote photoplethysmography

Figures

• 

  Figure 1. Prediction example. Predictions on the Subject1 of the UBFC Dataset.

• 

  Figure 2. The pyVHR pipeline at a glance. (A) The multi-stage pipeline of the pyVHR framework for BPM estimate through PSD analysis exploiting end-to-end DL-based methods. (B) The multi-stage pipeline for traditional approaches that goes through: windowing and patch collection, RGB trace computation, pre-filtering, the application of an rPPG algorithm estimating a BVP signal, post-filtering and BPM estimate through PSD analysis.

• 

  Figure 3. Comparison of the two implemented skin extraction methods. Output of the Convex-hull approach (A) and face parsing by BiSeNet (B) on a subject of the LGI-PPGI dataset (Pilz et al., 2018).

• 

  Figure 4. Landmarks automatically tracked by MediaPipe and correspondent patch tracking on a subject of the LGI-PPGI dataset (Pilz et al., 2018).

• 

  Figure 5. Patch tracking within a frame temporal window on a subject of the LGI-PPGI dataset (Pilz et al., 2018).

• 

  Figure 6. Predicted BVP signals. An example of estimated BVP signals on the same time window by four different methods. (A) POS. (B) GREEN. (C) CHROM. (D) PCA.

• 

  Figure 7. Estimated PSD. Estimated Power Spectral Densities (PSD) for the BVP signals plotted in Fig. 6. The BPM estimate, given by the maxima of the PSD, is represented by the blue dashed line. (A) POS. (B) GREEN. (C) CHROM. (D) PCA.

• 

  Figure 8. Distribution of BPM predictions by four methods on P patches. (A) POS. (B) GREEN. (C) CHROM. (D) PCA. Kernel Density Estimates (KDEs) of the predicted BPMs in a time window from P = 100 patches. The ultimate BPM prediction is given by the median (gold dashed line). The uncertainty estimate delivered by the Median Absolute Deviation (MAD) is shown by the golden band around the median. The blue dashed line represents the actual BPM.

• 

  Figure 9. Comparison of predicted vs ground truth BPMs using the patch-wise approach. Predicted BPM (blue) for the Subject1 of the UBFC Dataset. The uncertainty is plotted in shaded blue, while the ground truth is represented by the red line.

• 

  Figure 10. Per-frame time requirements. Average time requirements to process one frame by the Holistic and Patches approaches when using CPU vs. GPU accelerated implementations. The green dashed line represents the real-time limit at 30 frames per second (fps).

• 

  Figure 11. The graphical user interface. A screenshot of the graphical user interface (GUI) for online video analysis. The plot on the left shows the predicted BPMs, while on the right it is shown the processed video frames (captured with a webcam) with an example of the segmented skin and the tracked patches.

• 

  Figure 12. The assessment module at a glance. One or more datasets are loaded; videos are processed by the pyVHR pipeline while ground-truth BPM signals are retrieved. Predicted and real BPM are compared with standard metrics and the results are rigorously analyzed via hypothesis testing procedures.