Anatomically informed deep learning on contrast-enhanced cardiac magnetic resonance imaging for scar segmentation and clinical feature extraction

Figure 1. Distribution of enhanced myocardium regions for ground truth data. The spatial distribution of regions of enhanced myocardium is shown for 3 regions of the ventricle: basal (left), middle (center), and apical (right). The x-axes capture septal vs lateral location and the y-axes capture anterior vs posterior. The heat map quantifies the proportion of enhanced myocardium located in the respective region, averaged over all patients.

Figure 2. Conversion process of cine images to “late gadolinium enhancement (LGE)-like” images. (1) The original cine image is cropped/padded to a square and contrast-limited adaptive histogram equalization (CLAHE) is applied. (2) Cine images are further transformed by first generating a pseudo-enhancement (“LGE-like” enhanced myocardium) mask. (3) Pseudo-scar mask is randomly eroded and Gaussian filters are applied to realistically blur the edges. (4) Speckle noise is added to the image to resemble LGE noise. (5) An LGE cardiac magnetic resonance scan is sampled at random and a histogram match is performed.

Figure 3. ACSNet architecture consisting of 3 interconnected deep learning subnetworks. A: The first residual U-Net (ResU-Net) is used to identify and crop around the left ventricle (LV). B: The second network uses the tightly cropped image from panel A and the LV segmentation to further segment the LV into viable and enhanced myocardium. C: The third network is a convolutional autoencoder trained to encode (compress) and decode myocardial segmentation masks. D: Segmentations from the training set are encoded using the third network to form a latent space. The space is modeled as a Gaussian mixture model (GMM) and conditional resampling is performed to populate the space with anatomically correct samples (black dots). Predicted segmentations are encoded and the nearest-neighbors algorithm is used to return a perturbed, anatomically correct version (green dot) of the original (red dot). GMM image adapted from source.48 Additional details are presented in Methods.

Figure 4. Left ventricle and myocardium segmentation network architecture. The left ventricle region of interest network (LV Net.) identifies the main region of interest. The second network (MYO Net.) segments the myocardium by differentiating between viable and nonviable tissue represented by each of the 2 outputs. The networks differ by the number of filters, input image size, and number of outputs as indicated.

Figure 5. Autoencoder network architecture. The anatomical autoencoder is used as a postprocessing step that takes in myocardial masks and uses a series of convolutions and downsampling layers to create a 16-dimensional latent representation (left side). The decoder piece of the autoencoder (right side) re-creates the original myocardial segmentation from the latent representation.

Figure 6. Left ventricle (LV) region of interest (ROI) and myocardium segmentation results by region. Histograms of per-slice Dice scores are shown for 3 regions of the heart (rows from top to bottom: basal, middle, and apical). Columns represent the left ventricle (LV ROI) segmentation (A–C) and myocardium (MYO) segmentation (D–F). The averages are shown as solid vertical lines, and the dotted lines represent the 5th and 95th percentiles. LGE-CMR = cardiac magnetic resonance imaging with late gadolinium enhancement.

Figure 7. Scar segmentation results. Segmentations of enhancement regions from the myocardium segmentation network represent gray zone (yellow) and scar (red). The first row shows the original scan, the middle row shows the ground truth scar and gray zone segmentations, and the bottom row shows the predicted segmentations. LGE-CMR = cardiac magnetic resonance imaging with late gadolinium enhancement.

Figure 8. Scar and left ventricle (LV) volumes. LV (A) and scar (B) volume error is computed as the absolute error normalized by each respective volume. Each point represents the error in LV volume of a single segmented patient scan. The solid black line shows the mean.