Comparison of the performance between an AI-based vision transformer and human endoscopists in predicting the endoscopic and histologic activities of ulcerative colitis

Background Colonoscopy plays a vital role in assessing disease activity in ulcerative colitis (UC), and biopsy via colonoscopy helps to evaluate its histological activity. Endoscopists must report the endoscopic activity and rely on the biopsy results to predict the histological activity. Methods We aimed to develop a deep learning-based algorithm to evaluate the disease and histological activities of UC based on white-light endoscopic images obtained during the procedure in this research. A deep learning system for classifying the colonoscopic images for assessing the endoscopic and histological activities of UC patients was developed. Its performance was evaluated with an independent dataset. The system was utilized to analyze the captured video segments, and the results were compared with those of human endoscopists. Results A total of 375 video segments from 82 patients were utilized to develop the endoscopic and histological activity prediction assurance algorithm. Among the 375 video segments, 60%, 20%, and 20% were used for training, validation, and testing the proposed vision transformer (ViT) model, respectively. Moreover, four senior and six young endoscopists reviewed and scored the endoscopic and histological activities based on 77 testing video clips. The accuracies were 77.92%, 71.00%, and 83.12% for histological healing; and 74.35%, 72.51%, and 92.21% for complete mucosal healing (Mayo Endoscopic Score 0 vs 1–3), among senior endoscopists, junior endoscopists, and the ViT model, respectively. Conclusions Our novel deep learning-based model, based on endoscopic videos, was comparable to that of experienced endoscopists and surpassed that of young endoscopists in predicting histological remission and complete mucosal healing.