Automated abdominal adipose tissue segmentation and volume quantification on longitudinal MRI using 3D convolutional neural networks with multi-contrast inputs

Fig. 1. The network structures for A attention-based competitive dense (ACD) 3D U-Net and B 3D nnU-Net. The filter sizes are reported on top of each convolutional block. The full field-of-view volumetric opposed-phase echo image (TEOP), water, and fat images were combined as the multi-contrast inputs, and the 3D networks outputted segmentation masks for abdominal SAT and VAT concurrently. A In the ACD 3D U-Net, each blue box represents a competitive densely connected block. At the bottom layer, convolutional attention blocks shown in dark gray color were added to consider the more informative spatial and channel features. B A self-configuring 3D nnU-Net was trained based on a previously published framework. At each resolution step, 3D nnU-Net employed a convolution, followed by instance normalization and a leaky rectified linear unit (ReLU) activation function. Strided convolution and transposed convolution were used for downsampling and upsampling, respectively

Fig. 2. Input images (TEOP, fat, and water), reference annotations, and output abdominal SAT (white) and VAT (gray) segmentation masks from 3D U-Net, attention-based competitive dense (ACD) 3D U-Net, and 3D nnU-Net in two representative subjects. A Results for subject 1, a 75-year-old female with body-mass-index (BMI) of 29.0 kg/m2. B Results for subject 2, a 41-year-old female with BMI of 30.2 kg/m2. The regions misclassified by 3D U-Net were shown in zoomed insets and indicated by red arrows. The proposed ACD 3D U-Net and 3D nnU-Net were able to segment the corresponding regions (green arrows) with closer agreement to the reference annotations (Ref. Annot.). The highest DICE scores of the three CNNs were indicated with bold font

Fig. 3. False negative pixels (yellow) and false positive pixels (red) for the two representative subjects in Fig. 2 (A, B) using 3D U-Net, the proposed attention-based competitive dense (ACD) 3D U-Net, and 3D nnU-Net. The false negative and positive ratios are reported for the shown slices and the lowest values are highlighted with bold font

Fig. 4. Segmentation performance of the 3D U-Net, the proposed attention-based competitive dense (ACD) 3D U-Net, and 3D nnU-Net in terms of the 3D DICE-SAT (A), DICE-VAT (B), false negatives (FN) (C), and false positives (FP) (D) in the testing set of the first MRI (n = 182). The * indicates statistically significant differences in median scores using Benjamini-Hochberg’s procedure for multiple comparison. The color of * represents the better performing 3D CNN. Overall, ACD 3D U-Net and 3D nnU-Net performed similarly and were better than 3D U-Net. Median DICE-SAT (p < 0.001) and DICE-VAT (p < 0.001) were significantly higher for 3D nnU-Net and ACD 3D U-Net than 3D U-Net. ACD 3D U-Net achieved significantly lower median FN when compared to 3D U-Net and 3D nnU-Net (p < 0.001). ACD 3D U-Net produced lower maximum FP values than 3D U-Net and 3D nnU-Net

Fig. 5. Segmentation performance of the 3D U-Net, the proposed attention-based competitive dense (ACD) 3D U-Net, and 3D nnU-Net in terms of the 3D DICE-SAT (A), DICE-VAT (B), false negatives (FN) (C), and false positives (FP) (D) in the testing set of the second MRI (n = 182). The * indicates statistically significant differences in median scores using Benjamini-Hochberg’s procedure for multiple comparison. The color of * represents the better performing 3D CNN. Overall, ACD 3D U-Net and 3D nnU-Net performed similarly and were better than 3D U-Net. ACD 3D U-Net and 3D nnU-Net achieved significantly higher median DICE-SAT (p < 0.001) and DICE-VAT (p < 0.001) scores compared to 3D U-Net. ACD 3D U-Net achieved significantly lower median FN when compared to 3D U-Net and 3D nnU-Net (p < 0.001). 3D nnU-Net produced lower maximum FP values than 3D U-Net and ACD 3D U-Net

Fig. 6. The abdominal subcutaneous and visceral adipose tissue (SAT, VAT) volume agreement between the reference annotations and results from 3D U-Net, ACD 3D U-Net, and 3D nnU-Net. A, B Results from the testing set of the first MRI scan (n = 182). C, D Results from the testing set of the second MRI scan (n = 182). The 3D convolutional neural networks trained with the same 646 subjects from the first MRI. The linear regression equations as well as intra-class correlation coefficients (ICC) are reported in the figure legends. ICCU ICC value for the 3D U-Net. ICCACD ICC value for ACD 3D U-Net. ICCnn ICC value for 3D nnU-Net

Fig. 7. Correlation of abdominal SAT (A) and VAT (B) volume difference [mL] (between the first and second MRI scans) for results from 3D convolutional neural networks (3D U-Net [U], ACD 3D U-Net [ACD], and 3D nnU-Net [nn]) and reference annotations. Linear regression equations are shown