Pd30-11 External Validation for a Fully Automated Urine Cytology Support Artificial Intelligence System

You have accessJournal of UrologyBladder Cancer: Non-invasive II (PD30)1 May 2024PD30-11 EXTERNAL VALIDATION FOR A FULLY AUTOMATED URINE CYTOLOGY SUPPORT ARTIFICIAL INTELLIGENCE SYSTEM Masatomo Kaneko, Yuki Harada, Keisuke Tsuji, Divyangi Paralkar, Atsuko Fujihara, Kengo Ueno, Masaya Nakanishi, Eiichi Konishi, Tetsuro Takamatsu, Go Horiguchi, Satoshi Teramukai, Toshiko Ito-Ihara, Andre Luis Abreu, and Osamu Ukimura Masatomo KanekoMasatomo Kaneko , Yuki HaradaYuki Harada , Keisuke TsujiKeisuke Tsuji , Divyangi ParalkarDivyangi Paralkar , Atsuko FujiharaAtsuko Fujihara , Kengo UenoKengo Ueno , Masaya NakanishiMasaya Nakanishi , Eiichi KonishiEiichi Konishi , Tetsuro TakamatsuTetsuro Takamatsu , Go HoriguchiGo Horiguchi , Satoshi TeramukaiSatoshi Teramukai , Toshiko Ito-IharaToshiko Ito-Ihara , Andre Luis AbreuAndre Luis Abreu , and Osamu UkimuraOsamu Ukimura View All Author Informationhttps://doi.org/10.1097/01.JU.0001008848.77629.6f.11AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: To externally validate a fully automated urine cytology support artificial intelligence (AI) system to predict histological high-grade urothelial carcinoma (HGUC). METHODS: Urine cytology slides were collected from patients with suspicion of urothelial carcinoma at three institutions (IRB# ERB-C 1339-5, 1673-1, and S2020-60). Two board-certified cytotechnologists and a cytopathologist independently reviewed and labeled the slides according to the Paris system. An unsatisfactory slide was excluded from the analysis. Collected cytology slides were digitized for image analysis. A deep learning AI model was designed to automatically analyze the digitized slides, generate an assessment report which indicates the corresponding classification and lists image of the top 15 suspicion for HGUC cells, and support a pathologist in making a diagnosis. The performance in predicting histological HGUC from urine cytology slides was evaluated by the receiver operating characteristic (ROC). The optimal cutoff of the AI probability score was determined by the Youden index on the training dataset. The performance was compared to a board-certified pathologist by McNemar test. Statistically significant if p<0.05. RESULTS: A total of 676 urine cytology slides were collected. AI was trained by the randomly selected 240 slides from two institutions, and internally/externally validated by unseen 436 slides (341 internal and 95 external) collected before the histological confirmation by bladder biopsy. In total, 38% of the patients in the test group had histological HGUC. The area under the ROC curve (AUC) of AI was 0.81 in internal validation and 0.74 in external validation. The performance comparing AI vs pathologists were, for internal validation: sensitivity (59% vs 48%, p=0.01); specificity (86% vs 89%, 0.1); and accuracy (76% vs 74%, p=0.4); and for external validation: sensitivity (46% vs 22%, p=0.02); specificity (86% vs 95%, 0.1); and accuracy (71% vs 66%, p=0.3), respectively (Figure 1). CONCLUSIONS: A fully automated AI system predicts histological HGUC from urine cytology slides with higher sensitivity than board-certified pathologists in both internal and external validation. The AI system generates AI reports to support pathologists in making a diagnosis. Download PPT Source of Funding: This work was supported by Japan Society for the Promotion of Science KAKENHI Grant Number JP23K08783 © 2024 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 211Issue 5SMay 2024Page: e628 Advertisement Copyright & Permissions© 2024 by American Urological Association Education and Research, Inc.Metrics Author Information Masatomo Kaneko More articles by this author Yuki Harada More articles by this author Keisuke Tsuji More articles by this author Divyangi Paralkar More articles by this author Atsuko Fujihara More articles by this author Kengo Ueno More articles by this author Masaya Nakanishi More articles by this author Eiichi Konishi More articles by this author Tetsuro Takamatsu More articles by this author Go Horiguchi More articles by this author Satoshi Teramukai More articles by this author Toshiko Ito-Ihara More articles by this author Andre Luis Abreu More articles by this author Osamu Ukimura More articles by this author Expand All Advertisement PDF downloadLoading ...