A task-unified network with transformer and spatial–temporal convolution for left ventricular quantification

Figures

• 

  Figure 1. Two categories methods have existed in left ventricular quantification domain. (a) Segmentation-based methods compute indices from the segmented result which requires strong prior information and user interaction. (b) Existing direct regression methods of cardiac indices quantification. When the labeled image is not available, direct methods-regression without segmentation step have grown in popularity in cardiac LV indices estimation.

• 

  Figure 2. Schematic illustration of LV indices for short-axis cardiac MRI. (a) The LV and Myocardium cavity areas are shown with blue and pink color. (b) LV cavity directional dimensions with black arrows. (c) Six myocardial regional-wall thicknesses (RWT), namely anterolateral (AL), inferolateral (IL), inferior (I), inferoseptal (IS), anterior (A), inferoseptal (IS) and anteroseptal (AS). (d) The cardiac phase (systole or diastole).

• 

  Figure 3. Overview of the proposed unified framework which contains segmentation path and regression path.

• 

  Figure 4. After pre-processing, we the stack the input image and its corresponding groundtruth to highlight the LV cavity and LV myocardium. From the figure, we can see the variation of shape, contrast and density in cardiac MRI. It is a great challenge for segmentation and quantification.

• 

  Figure 5. Example of predictions by our model for a random 20 frames: The rows of (a) and (d) are input images and corresponding myocardial, (b) and (e) are the myocardial predictions from our unified network. The rows of (c) and (f) are the difference of target myocardial area between ground truth and predicted segmentation, where red represents the under segmented regions and blue indicates the over segmented regions.

• 

  Figure 6. Example of comparative segmentation results on LV-Quan validation data. The first row are the raw input images and its corresponding ground truth. From the second to the fifth row indicate the predictions by comparative ablation models.

• 

  Figure 7. Example of comparative segmentation results on LV-09 testing dataset. The rows from top to bottom indicate the image slices from four pathological group: Heart Failure with Ischemia (HF-I), Heart Failure without Ischemia (HF-NI), Hypertrophic endocardium (HYP), Normal (NOR). The columns from left to right indicate predictions of comparative methods, GroundTruth and raw image data respectively.

• 

  Figure 8. The LV quantification indices results predicted by our method on the testing data.

• 

  Figure 9. The LV quantification indices results predicted by our method on the testing data.

• 

  Figure 10. Examples of LV metrics predicted by our unified network for a random patient during a cardiac cycle (20 frames). In the following line charts, the predicted results are shown with the dotted line and the corresponding ground truth values are displayed with the solid line in the same marker.

• 

  Figure 11. Visualization of prediction and output probability map. The rows marked W/O transformer indicates the model without using transformer module, while with transformer indicates model using transformer module.