Key result
Spatial-Frequency-Temporal Transformer outperforms state-of-the-art models in vision-based heart rate estimation.
Why the study?
Vision-based heart rate estimation in adverse situations, such as skin tone changes, arbitrary face movements, and complex backgrounds, remains challenging.
The SFTT model provides a robust and generic approach for vision-based heart rate estimation, outperforming existing state-of-the-art models across multiple datasets.
Vision-based Heart Rate (HR) estimation in adverse situations, such as changes in skin tone, arbitrary face movements, and complex backgrounds, etc., is challenging. Unlike state-of-the-art models that use color and spatial-temporal information, the present work exploits a Spatial-Frequency-Temporal Transformer (SFTT) for heart rate estimation. For extracting multi-scale contextual features, we propose an end-to-end transformer that encodes contextual information through a pyramidal structure-based approach. Furthermore, to strengthen the features, the proposed model introduces a new attention approach that performs mutual-sharing operations between spatial-temporal and frequency-temporal domains in an end-to-end fashion. Experimental results on four standard datasets, namely UBFC-rPPG, VIPL-HR, OBF, and MMSE-HR, show that the proposed model is generic and invariant to the aforementioned challenges. Further, a comparative study with the state-of-the-art models demonstrates the effectiveness of the proposed method over the existing methods on all four benchmark datasets. Besides, experiments on cross-dataset validation show that the proposed method is reliable and robust.
No takes yet. Share an insight, caveat, or question.
Dey et al. (2025) studied Heart Rate Estimation. Spatial-Frequency-Temporal Transformer (SFTT) vs. State-of-the-art models was evaluated on Heart rate estimation performance. The Spatial-Frequency-Temporal Transformer (SFTT) demonstrated effectiveness and robustness for vision-based heart rate estimation across four benchmark datasets compared to state-of-the-art models.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: