Key result
MFF-Net achieved better performance on heart rate and heart rate variability estimation with lower computational burden compared to state-of-the-art methods across three benchmark datasets.
Why the study?
Existing deep learning models for rPPG cannot effectively eliminate illumination variation and motion artifacts, and their high computational demands limit real-world use.
The proposed MFF-Net offers a lightweight, accurate deep learning approach for remote heart rhythm monitoring from facial videos, potentially enabling real-world applications.
Remote photo-plethysmography (rPPG) is a useful camera-based health motioning method that can measure the heart rhythm from facial videos. Many well-established deep learning models can provide highly accurate and robust results in measuring heart rate (HR) and heart rate variability (HRV). However, these methods are unable to effectively eliminate illumination variation and motion artifact disturbances, and their substantial computational resource requirements significantly limit their applicability in real-world scenarios. Hence, we propose a lightweight multi-frequency network named MFF-Net to measure heart rhythm via facial videos in a short time. Firstly, we propose a multi-frequency mode signal fusion (MFF) mechanism, which can separate the characteristics of different modes of the original rPPG signals and send them to a processor with independent parameters, helping the network recover blood volume pulse (BVP) signals accurately under a complex noise environment. In addition, in order to help the network extract the characteristics of different modal signals effectively, we designed a temporal multiscale convolution module (TMSC-module) and spectrum self-attention module (SSA-module). The TMSC-module can expand the receptive field of the signal-refining network, obtain more abundant multiscale information, and transmit it to the signal reconstruction network. The SSA-module can help a signal reconstruction network locate the obvious inferior parts in the reconstruction process so as to make better decisions when merging multi-dimensional signals. Finally, in order to solve the over-fitting phenomenon that easily occurs in the network, we propose an over-fitting sampling training scheme to further improve the fitting ability of the network. Comprehensive experiments were conducted on three benchmark datasets, and we estimated HR and HRV based on the BVP signals derived by MFF-Net. Compared with state-of-the-art methods, our approach achieves better performance both on HR and HRV estimation with lower computational burden. We can conclude that the proposed MFF-Net has the opportunity to be applied in many real-world scenarios.
No takes yet. Share an insight, caveat, or question.
Zhuang et al. (2024) studied Heart rhythm measurement. MFF-Net (lightweight multi-frequency network) vs. State-of-the-art methods was evaluated on Heart rate (HR) and heart rate variability (HRV) estimation performance. MFF-Net achieved better performance on heart rate and heart rate variability estimation with lower computational burden compared to state-of-the-art methods across three benchmark datasets.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: