Artificial intelligence-based multimodal model for the identification of ulcerative colitis with concomitant cytomegalovirus colitis.

Ulcerative colitis (UC), a chronic immune-mediated colon inflammation, impacts patients' quality of life. Immunosuppressive-treated UC patients are prone to opportunistic infections like cytomegalovirus (CMV) infection, which exacerbates UC, causes steroid resistance, and elevates surgery and mortality risks. Identifying CMV colitis from UC exacerbation is difficult due to overlapping symptoms and low biopsy detection rates. To develop an artificial intelligence (AI)-based multimodal model for early identification of UC with concomitant CMV colitis. This was a retrospective diagnostic study. A total of 174 moderate to severe UC patients (87 with CMV colitis) from 2015 to 2023 in Peking Union Medical College Hospital were enrolled retrospectively. A total of 3345 colonoscopy images were collected. The dataset was split into training (70%) and testing (30%) sets. A multimodal dynamic affine transformation (DAFT) model integrating clinical biomarkers and endoscopic images was constructed, along with ResNet and SeNet models. Model performance was evaluated using accuracy, sensitivity, specificity, positive and negative predictive values from the confusion matrix. UC patients with CMV colitis had distinct clinical characteristics. The multimodal DAFT model outperformed ResNet and SeNet in distinguishing UC with CMV colitis, with higher accuracy (0.91), sensitivity (0.87), and specificity (0.93). AI application offers a promising way to enhance early identification of UC with CMV colitis. The multimodal model combining clinical and endoscopic data can assist clinicians in accurate and timely diagnosis.