Evaluation of AI for prostate cancer detection in biparametric-MRI screening population data

Abstract Objective The goal of this study was to curate a prostate MRI dataset from a screening population and to train and evaluate a deep-learning segmentation method on the same data. Materials and methods An artificial intelligence (AI) system, based on a deep-learning-based segmentation model (nnU-Net method), was trained and evaluated with MRI data from a prostate cancer screening population (G2-trial). The goal of the AI was to detect clinically significant prostate cancer (csPC), defined as International Society of Urological Pathology (ISUP) grade 2 or higher. The AI system was compared to the performance of radiologists using PI-RADS v2 evaluation metrics. Histopathology was used as the reference standard in the dataset. To better verify negative cases, 288 men were subject to systematic biopsies regardless of MRI findings, and all men had at least 3 years of follow-up. Results A total of 1354 MRI examinations in 1254 men with a median age of 58 years (range 50–63 years) were randomly divided into a training set (1086 examinations) and a test set (268 examinations). The resulting area under the receiver operating characteristic curve (AUROC) was 0.83 (95% CI 0.73–0.92) for the AI system; however, with significantly lower specificity at matched sensitivity levels compared to radiologists. Conclusion A prostate MRI dataset from a screening population with histological confirmation was curated and evaluated with AI. The neural network trained and tested on this data produced lower specificities than the radiologists. Key Points Question Does an AI system trained in a screening cohort perform as well as radiologists? Findings An AI trained on screening data achieved an AUROC of 0.83 (95% CI 0.73–0.92) with lower specificity at the same sensitivity levels as radiologists. Clinical relevance An AI system trained in a screening population has lower specificity than radiologists using PI-RADS v2. Graphical Abstract