Automated cartilage and meniscus segmentation of knee MRI with conditional generative adversarial networks

PURPOSE: Fully automatic tissue segmentation is an essential step to translate quantitative MRI techniques to clinical setting. The goal of this study was to develop a novel approach based on the generative adversarial networks for fully automatic segmentation of knee cartilage and meniscus. THEORY AND METHODS: Defining proper loss function for semantic segmentation to enforce the learning of multiscale spatial constraints in an end-to-end training process is an open problem. In this work, we have used the conditional generative adversarial networks to improve segmentation performance of convolutional neural network, such as UNet alone by overcoming the problems caused by pixel-wise mapping based objective functions, and to capture cartilage features during the training of the network. Furthermore, the Dice coefficient and cross entropy losses were incorporated to the loss functions to improve the model performance. The model was trained and tested on 176, 3D DESS (double-echo steady-state) knee images from the Osteoarthritis Initiative data set. RESULTS: The proposed model provided excellent segmentation performance for cartilages with Dice coefficients ranging from 0.84 in patellar cartilage to 0.91 in lateral tibial cartilage, with an average Dice coefficient of 0.88. For meniscus segmentation, the model achieves 0.89 Dice coefficient for lateral meniscus and 0.87 Dice coefficient for medial meniscus. The results are superior to previously published automatic cartilage and meniscus segmentation methods based on deep learning models such as convolutional neural network. CONCLUSION: The proposed UNet-conditional generative adversarial networks based model demonstrated a fully automated segmentation method with high accuracy for knee cartilage and meniscus.