HRVCam: robust camera-based measurement of heart rate variability

Figures

• 

  Fig. 1. (a) Comparison of the normalized power spectral density of the video iPPG signal and the ground truth cPPG signal. (b) IBI estimates from iPPG compared with the ground truth for different BWs. The estimates are most accurate for the BW of 1 Hz (middle figure). (c) Trade-off between losing HRV information and rejecting motion interference by comparing the error in RMSSD estimate across different BWs.

• 

  Fig. 2. (b) On using a large BW and including the information in the sidebands of the fundamental harmonic, the estimates of HRV RMSSD are more accurate and follow ground truth more closely. (a) HRV information is in the sidebands of the fundamental harmonic because of quasi-periodicity of pulse signal.

• 

  Fig. 3. Flowchart of HRVCam: iPPG signals extracted from raw videos is divided into epochs. Step 1: iPPG signal passed through an adaptive bandpass filter that estimates the cut-off frequencies of the filter based on the noise structure in the iPPG signal. The bandpass filter filters out the fundamental harmonic signal. Step 2: Discrete energy separation algorithm was used to extract out the instantaneous frequency. Step 3: the instantaneous frequency is converted to interbeat intervals by reconstructing the FM signal.

• 

  Fig. 4. Typical face video-based iPPG signal acquisition system. Participants performed different tasks such as sitting still, reading a facts page, watching a scary movie trailer, and talking about a personal experience in front of the camera.

• 

  Fig. 5. SNR of the iPPG signal for 60-s video computed with chrominance and distancePPG. Each scenario consists of 32 videos. The number of data point for low SNR regions is higher for the talking scenario.

• 

  Fig. 6. Scatterplot comparison of HRV metrics (a) RMSSD and (b) SDNN derived from iPPG with HRVCam, CDM, and Peak NC-VT methods, respectively, against ground truth pulse oximeter for different skin tones. Note that different y axis scales are used for the methods to accommodate all the points).

• 

  Fig. 7. Bland–Altman plot of HRV metrics (a) RMSSD and (b) SDNN derived from iPPG with HRVCam, CDM, and Peak NC-VT methods, respectively, against ground truth pulse oximeter for different skin tones.

• 

  Fig. 8. IBI estimates from HRVCam and Peak NC-VT against ground truth pulse oximeter (pulse ox) measurements for (a) light skin tone and (b) dark skin tone. The performance of HRVCam and Peak NC-VT is similar for light skin tone. However, for dark skin tone, IBI estimates from HRVCam follow ground truth more closely than Peak NC-VT. CDM estimates are smoother than HRVCam and Peak NC-VT.

• 

  Fig. 9. Scatterplot comparison of HRV metrics (a) RMSSD and (b) SDNN derived from iPPG with HRVCam, CDM, and Peak NC-VT methods, respectively, against ground truth pulse oximeter for different motion scenarios. Note that different y axis scales are used for the methods to accommodate all the points).

• 

  Fig. 10. Bland–Altman plot of HRV metrics (a) RMSSD and (b) SDNN derived from iPPG with HRVCam, CDM, and Peak NC-VT methods, respectively, against ground truth pulse oximeter for different motion scenarios.

• 

  Fig. 11. IBI estimates from HRVCam, Peak NC-VT, CDM, and ground truth pulse oximeter for different motion scenarios (a) reading a website, (b) watching videos, and (c) talking. HRVCam follows ground truth pulse oximeter closely in reading and watching scenarios. In talking, the performance of HRVCam suffers. However, HRVCam provides significantly better estimates than Peak NC-VT and CDM.

• 

  Fig. 12. (a) Distribution of mae in RMSSD across different SNR obtained from iPPG signal computed for different activities. For lower SNR [calculated with Eq. (24)] videos, HRVCam shows large improvements. (b) Distribution of error in average heart rate across SNR