End-to-end non-invasive ECG signal generation from PPG signal: a self-supervised learning approach

Figures

• 

  FIGURE 1. First two columns shows the screenshots of the VR game (Half Life-Alyx) that participants played during data collection. Last column shows an equipped participant and the respective sensors used during the experiments in detail.

• 

  FIGURE 2. The overall ECG generator architecture model which consist of three branches: 1- The convolutional encoder (left-side) which receives different time length of PPG signal as input extracts spatial features. 2- The Transformer encoder is concatenated between convolution encoder-decoder with causal self-attention for long-range dependency modeling (bottom-side). 3- The convolutional decoder (right-side) up-samples feature maps and outputs the ECG signal with corresponding length. The grey arrows (middle-side) shows skip connections between the corresponding layers of the convolutional encoder-decoder. The feature maps from the encoder are concatenated with the upsampled feature maps in the decoder. This allows the network to combine high-resolution features from early layers with the high-level abstract features learned in deeper layers.

• 

  FIGURE 3. The overall scheme illustrates the implementation of self-supervised learning with the discriminator (shown in a dashed rectangle) and its subsequent use as a frozen network for fine-tuning our GAN model: 1- Top-side depicts the general steps of the self-supervised learning process (downstream task) using the discriminator. Transformed signals, generated from three datasets, are input into the discriminator to train it across six different tasks. 2- Down-side illustrates the partially frozen discriminator architecture, which is employed during the fine-tuning stage to distinguish between real and generated ECG signals.

• 

  FIGURE 4. Transformed two samples of ECG signals from two participants for pretext tasks of self-supervised learning. Noise addition, scaling, temporal inversion, permutation, negation and time-warping were implemented to each signals, separately.

• 

  FIGURE 5. The impact of the gradient penalty coefficient (factor) λ as hyperparameter on the training of the discriminator on original dataset. It addresses the mode collapse issue and strengthen the model stability during training. It clearly demonstrate that the absence of the gradient penalty (λ=0) leads to notable instability in the training process, underscoring the importance of this parameter in ensuring robust and reliable model performance.

• 

  FIGURE 6. We present four distinct ECG samples (with different colors) generated by our proposed best performed Transformer-based GAN model on Who is Alyx? dataset. These four samples illustrate cases with minimal motion distortion, providing a clear representation and demonstrating the robust generation capability of the model for ECG signals. For each PPG sample (green color), corresponding real ECG (blue color), generated ECG (yellow color) and overlayed (original and generated) ECG signal (blue + yellow color) samples are displayed, respectively.

• 

  FIGURE 7. We present four distinct ECG samples (with different colors) generated by our proposed best performed Transformer-based GAN model on Who is Alyx? dataset. These four samples specifically highlight cases with motion artifacts, demonstrating the model’s performance under challenging conditions involving signal distortion. For each PPG sample (green color), corresponding real ECG (blue color), generated ECG (yellow color) and overlayed (original and generated) ECG signal (blue + yellow color) samples are displayed, respectively.

• 

  FIGURE 8. We present four distinct ECG samples (with different colors) generated by our proposed best performed Transformer-based GAN model on MIMIC III dataset. Comparing to Who is Alyx? dataset, MIMIC III has less motion artefacts and emotional changes. For each PPG sample (green color), corresponding real ECG (blue color), generated ECG (yellow color) and overlayed (original and generated) ECG signal (blue + yellow color) samples are displayed, respectively.

• 

  FIGURE 9. Two-dimensional t-SNE projections of learned feature embeddings for 200 real and 200 generated ECG samples for MIMIC III (left) and Who is Alyx (right) datasets.

• 

  FIGURE 10. Confusion matrices illustrating AF detection performance on the generated ECG test set for the CNN + LSTM and Bid-LSTM models.

• 

  FIGURE 11. Qualitative comparison of real and generated ECG signals for two participants (AF and non-AF) from the MIMIC PERform AF Dataset. Top row: Representative 30-s segments of AF and non-AF ECG signals. Middle row: Baseline fluctuation profiles of real and generated ECGs from the full 20-min recordings, shown with mean values and standard deviations. Bottom row: Power spectral density (PSD) plots for AF and non-AF ECG signals, highlighting the 0.05–9 Hz frequency band relevant for baseline characterization and AF detection.