Digitizing paper ECGs at scale: an open-source algorithm for clinical research

Figures

• 

  Fig. 1. Shows the digitization pipeline for a paper ECG captured using a mobile phone. Top left: Mobile phone photo of a printed ECG. Top right: Segmented image showing detected ECG waveform and grid structure. Bottom left: Perspective-corrected and cropped ECG aligned. Bottom right: Final extracted waveforms in standardized format for archiving and downstream processing.

• 

  Fig. 2. Shows the necessary shift to align each lead with the ground truth. In general, smaller shifts are needed for images captured with a flatbed scanner, and for all three devices, no shift is the most common.

• 

  Fig. 3. Digitization metrics across devices. Leftmost panes show pairwise scatter plot of SNR per lead, for images captured with the three different devices. Rightmost panes show the distributions of signal quality metrics for ECG recordings captured using three devices (Scanner, iPhone, OnePlus). Using the scanner results in better average results across all three metrics. The two phones perform similarly, with OnePlus resulting in slightly better results.

• 

  Fig. 4. Lead II as found in the original recording and after digitization of high-resolution scanner images using our proposed algorithm. Five different ECGs are selected based on their SNR. In the lowest-SNR examples (bottom two), a slight baseline drift appears in the scanned-digitized traces, possibly due to the error margins in detecting the rotation of the paper.

• 

  Fig. 5. Lead II as found in the original recording and after digitization of images taken with an iPhone Pro Max 16 using our proposed algorithm. Five different ECGs are selected based on their SNR. The signals are not always perfectly aligned in time, resulting in lower SNR as compared to scanner images.

• 

  Fig. 6. Lead II as found in the original recording and after digitization of images taken with a OnePlus Nord CE 2 using our proposed algorithm. Five different ECGs are selected based on their SNR. The signals are not always aligned in time, resulting in lower SNR as compared to scanner images.

• 

  Fig. 7. Three examples of signals with significant numbers of NaNs in them, across the three devices. Instead of rejecting the images, the leads are left blank, and the user can decide what to do with them.

• 

  Fig. 8. The digitization pipeline consists of semantic segmentation, that separates gridlines, ecg signal, text and background, followed by perspective correction and cropping. These first two modules produce an image where the gridlines on the paper are aligned with the image. The last part converts the aligned image to 1D signals, identifies the lead layout and outputs a structured time series in the desired sample rate.

• 

  Fig. 9. The left image represents a paper with gridlines. Parallel lines viewed at an angle in the image domain end up, to a good approximation, on a line in the angle-radius domain. Sets of lines that are parallel in the image domain are mapped to vertical lines in the angle-radius domain. The green line in the angle-radius domain maps to the green star in the angle-angle domain. Note that the θ-values where the line exits the plot correspond to the coordinates in the angle-angle domain.

• 

  Fig. 10. Steps to identify the perspective in a photograph of a paper ECG. The leftmost panel shows the gridlines as detected by the neural network. The middle panel shows the grid transformed into the angle-radius domain using the Hough transform. Finally, the rightmost panel illustrates the domain where the lines of maximum variance are identified. Note that the maxima are much more pronounced in this domain and correspond to the vertical and horizontal gridlines. We encourage the reader to zoom in on the middle panel at around 0 and π/2.

• 

  Fig. 11. The left pane shows the autocorrelation for a column-sum signal for lags between 0 and 220 pixels. The left pane shows the template that is matched to the autocorrelation function.

• 

  Fig. 12. Example of a solution to a linear sum assignment problem. Note that three of the leftmost endpoints and three of the rightmost endpoints are unmatched, illustrating that wrapped matches are dropped. Further, note that the connected component in the top right corner is matched with itself. Self-connections are later rejected as noise.