Echocardiography-based AI for detection and quantification of atrial septal defect

Figures

• 

  Figure 1. Work flow of the ensemble model. Step 1: raw echocardiographic videos are separated for classification of views (red box). Step 2: disease detection models use different views to detect the presence of ASD (orange rectangles). Step 3: if ASD is present (denoted by “yes”), metrics associated with severity of ASD are assessed (blue rectangles). DL, deep learning; ASD, atrial septal defect.

• 

  Figure 2. Example of manual segmentation. Green dots were manually labeled as the endpoints of atrial septum and defect with the open-source program LabelMe. (A) Modified apical four-chamber view (A4C). (B) Parasternal short-axis view (PSAX). (C) Subxiphoid sagittal view (SC2A). (D) Subxiphoid four-chamber view (SC4C).

• 

  Figure 3. Detailed description of model architecture. (A) ASD detection neural network architecture, based on Resnet-18, combined with ASPP to increase visual field. The output of the model is the probability of ASD between 0 and 1. (B) ASD quantification neural network architecture, combining spatial convolution and temporal transformer, had three auxiliary outputs and one output from coarse to fine. Each auxiliary output is added to the feature map as an additional feature. (C) Structure diagram of temproal transformer. Each frame is encoded into a feature token by CNN. Self-attention could realize the long-distance dependence between frames. q, query; k, key; v, value.

• 

  Figure 4. The performance of ASD detection model in the external dataset. The performance of composite classifier model (red curve) had an AUROC of 0.92. Abbreviations as in Figure 1.

• 

  Figure 5. The output of quantification model in different views. The first row, one example test image of patients with ASD is shown respectively in A4C, PSAX, SC2A and SC4C view; orange and blue dots are the endpoints of defect and septum predicted by DL model. The second row, corresponding curves showing the variation of defect and septum in the video. Abbreviations as in Figure 1.

• 

  Figure 6. The output of quantification model with different confidence levels. The first row, one example test image of A4C view is shown respectively with the confidence of 0.9, 0.7, 0.5 and 0.3. The second row, corresponding curves showing the variation of defect and septum in the video.

• 

  Figure 7. Comparisons of quantitative metrics derived from the deep learning (DL) algorithm and physician based on bland and altman analysis. Bland-Altman plots compare automated and manual measurements for septum length and defect size in A4C, PSAX, SC2A and SC4C view. Abbreviations as in Figure 1.