External validation of a machine learning-based classification algorithm for ambulatory heart rhythm diagnostics in pericardioversion atrial fibrillation patients using smartphone photoplethysmography: the SMARTBEATS-ALGO study

Figures

• 

  Figure 1. Simultaneous smartphone-PPG and single-lead ECG recordings with CORAI Heart Monitor and KardiaMobile. (A) The single-lead ECG device KardiaMobile was attached to the back of the smartphone case on an iPhone 7. (B) Shows the user handling and finger placements when performing a simultaneous smartphone-PPG recording with the CORAI Heart Monitor and single-lead ECG recording. The study participants performed both type of recordings simultaneously by placing a fingertip over the camera lens of the smartphone, at the same time as placing other fingers from both hands on the electrodes of the single-lead ECG device. (C, E, G) The CORAI PPG Report showing smartphone-PPG recordings with automatic heart rhythm diagnosis from the machine learning algorithm, it’s estimated degree of certainty of the diagnosis and the calculated signal quality of the PPG tracing. (D, F, H) The user interface in the software application and ECG reports used during manual reading for the single-lead ECG recordings. The reader would select the heart rhythm diagnosis for the shown ECG recording. For the recordings in (C and D) the heart rhythm was sinus rhythm, in (E and F) it was atrial fibrillation, and in (G…

• 

  Figure 2. Training and external validation of the machine learning algorithm. The automatic machine learning algorithm was trained on PPG recordings in the separate training cohort. Heart rhythm determined by manual reading of the PPG recordings was used to train a support vector machine (SVM) classifier. The diagnostic performance of the trained SVM classifier was evaluated with PPG recordings in the external validation cohort compared with the heart rhythm from the manually interpreted simultaneous single-lead ECG recordings. AF, atrial fibrillation; AFL, atrial flutter; ML, machine learning; PPG, photoplethysmogram; SR, sinus rhythm.

• 

  Figure 3. Study participant flow chart in the external validation cohort. The most common medical reason for DCCV cancellation was non-compliance to anticoagulation therapy. Logistical reasons for screening failure included insufficient availability of study devices, the patients having the DCCV procedure before they could be asked for participation, and inability to return the study device. AF, atrial fibrillation; AFL, atrial flutter; DCCV, direct current cardioversion; SR, sinus rhythm.

• 

  Figure 4. Overall diagnostic performance. This figure shows the diagnostic performance of the automatic machine learning algorithm that classifies the heart rhythm from smartphone-PPG recordings made with CORAI Heart Monitor compared with manual Reading of the simultaneous single-lead ECG recordings with KardiaMobile, with AFL recordings excluded (left) and with AFL recordings included (right). Example recordings of FP and FN classification outputs are available in Supplementary material online, Figure S7. AF, atrial fibrillation; AFL, atrial flutter; ECG, electrocardiogram; FN, false negatives; FP, false positives; ML, machine learning; PPG, photoplethysmogram; SR, sinus rhythm.

• 

  Figure 5. Signal quality of the smartphone-PPG recordings made in the external validation cohort (top) and in the training cohort (bottom) as determined by the automatic signal quality algorithm. This figure shows the signal quality of the smartphone-PPG recordings for each of the study participants. The proportion of smartphone-PPG recordings with sufficient quality to make a heart rhythm diagnosis, as decided by the automatic signal quality algorithm, are shown on the y-axis. The study participants were given automatic real-time feedback of the user handling by the CORAI Heart Monitor application during the recordings to help preserve the signal quality of the recordings. A revision to this automatic real-time feedback improved the signal quality of the smartphone-PPG recordings further substantially. The revised real-time feedback was available for the final 84 study participants, #197 to #280, in the external validation cohort, and for all the participants in the training cohort. PPG, photoplethysmogram.

• 

  Figure 6. Algorithmic certainty of the made heart rhythm diagnosis for the PPG recordings in the external validation cohort, excluding AFL recordings, shown as the proportion having high diagnostic certainty for each study participant and compared before and after the update to the automatic real-time user feedback. PPG, photoplethysmogram.

• 

  Figure 7. Visualization of the smartphone-PPG recordings included in the training cohort (top-left) and the external validation cohort (top-right), excluding AFL recordings, using a t-SNE algorithm that reduces each PPG recording to a single point in 2D space clustering similar PPG recordings together, showing separation between PPG recordings where the heart rhythm is SR and where it is AF. Diagnostic performance of the ML-based automatic algorithm trained on PPG recordings in the training cohort and externally validated on PPG recordings in the external validation cohort are visualized (bottom-right) and the prediction certainty of the recordings is shown (bottom-left). AF, atrial fibrillation; FN, false negatives; FP, false positives; ML, machine learning; PPG, photoplethysmogram; SR, sinus rhythm; TN, true negatives; TP, true positives.