Key result
Smartphone video-PPG deep learning accurately estimates vitals and reconstructs single-lead ECGs for rapid self-testing.
Why the study?
There is a need for ubiquitous, low-cost, non-invasive tools for rapid and continuous monitoring of vital signs reflecting overall health status in the post-covid19 era.
Can deep learning models accurately estimate pulse rate, SpO2, respiratory rate, and reconstruct a single-lead ECG from smartphone video-PPG data?
Can deep learning models accurately estimate pulse rate, SpO2, respiratory rate, and reconstruct a single-lead ECG from smartphone video-PPG data?
A smartphone-based deep learning approach using video-PPG can potentially estimate vital signs and reconstruct a single-lead ECG without additional hardware, enabling accessible self-monitoring.
May support accessible smartphone vital monitoring; leaves open prospective validation before clinical adoption.
In the post-covid19 era, every new wave of the pandemic causes an increased concern/interest among the masses to learnmore about their state of well-being. Therefore, it is the need of the hour to come up with ubiquitous, low-cost, non-invasivetools for rapid and continuous monitoring of body vitals that reflect the status of one’s overall health. In this backdrop, this workproposes a deep learning approach to turn a smartphone—the popular hand-held personal gadget—into a diagnostic tool tomeasure/monitor the three most important body vitals, i.e., pulse rate (PR), blood oxygen saturation level (aka SpO2), andrespiratory rate (RR). Furthermore, we propose another method that could extract a single-lead electrocardiograph (ECG)of the subject. The proposed methods include the following core steps: subject records a small video of his/her fingertip byplacing his/her finger on the rear camera of the smartphone, and the recorded video is pre-processed to extract the filteredand/or detrended video-photoplethysmography (vPPG) signal, which is then fed to custom-built convolutional neural networks(CNN), which eventually spit-out the vitals (PR, SpO2, and RR) as well as a single-lead ECG of the subject. To be precise,the contribution of this paper is two-fold: 1) estimation of the three body vitals (PR, SpO2, RR) from the vPPG data usingcustom-built CNNs, vision transformer, and most importantly by CLIP model (a popular image-caption-generator model); 2) anovel discrete cosine transform+feedforward neural network-based method that translates the recorded video-PPG signal to asingle-lead ECG signal. The significance of this work is two-fold: i) it allows rapid self-testing of body vitals (e.g., self-monitoringfor covid19 symptoms), ii) it enables rapid self-acquisition of a single-lead ECG, and thus allows early detection of atrialfibrillation (abormal heart beat or arrhythmia), which in turn could enable early intervention in response to a range of cardiovascular diseases,and could help save many precious lives. Our work could help reduce the burden on healthcare facilities and could lead toreduction in health insurance costs.
No takes yet. Share an insight, caveat, or question.
Mehmood et al. (2023) studied Healthy subjects and public dataset patients (n=623). Smartphone video-PPG with deep learning models (Vitals-Net, P2E-Net) vs. Ground truth from clinical pulse oximeter and reference ECG was evaluated on Mean absolute error (MAE) of estimated vital signs (HR, SpO2, RR) and reconstructed ECG. A novel deep learning approach utilizing smartphone video-PPG data successfully estimated pulse rate, blood oxygen saturation, and respiratory rate, and reconstructed a single-lead ECG, enabling rapid self-testing of body vitals.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: