P0518Assessing the Impact of Biopsy Artifacts on AI-Based Endoscopic Scoring for Ulcerative Colitis

Background UC-SCALE is an Artificial Intelligence (AI) algorithm that predicts the Mayo Clinic Endoscopic Subscore (MCES 0-3) at different anatomical regions (rectum, sigmoid and descending colon) to assess mucosal healing in Ulcerative Colitis (UC) [1]. For such AI tools to be reliable in practice, they must be robust to common procedural artifacts. Biopsy acquisition is a key source of such artifacts, as the use of forceps and subsequent bleeding can obscure mucosal features. This study investigates whether these biopsy-related events confound UC-SCALE’s predictions and impact its diagnostic accuracy. Methods A total of 1,487 unique frames were randomly selected from sigmoidoscopy videos in phase III etrolizumab clinical trials [2] and annotated for forceps presence. About 7.4% (110 frames) contained forceps. Hibiscus II (236 frames) and Hibiscus I (253 frames) were used for validation and testing respectively, while the remaining 998 frames formed the training set. A CNN-based forceps detector achieved F1-score of 0.87 in the test set. Applied to over one million unlabeled frames, it predicted forceps in 9% of cases, distributed evenly across MCES but more frequent in sigmoid sections.To assess the impact of biopsy artifacts on overall model performance and prediction stability, we compared two models. The baseline (biopsy-present training, BP-Tr) used all frames, while the second was retrained on data excluding forceps frames and the subsequent 30 seconds (biopsy-cleared training, BC-Tr). Both models were evaluated on the complete test set (biopsy-present evaluation, BP-Ev) and a cleaned version (biopsy-cleared evaluation, BC-Ev). We evaluated performance on two subsets: ‘All sections’ and ‘Forceps-impacted sections,’ with the latter including only anatomical sections containing one or more artifact frames. Results Both models produced nearly identical MCES prediction performance across all test conditions and metrics (Table 1). Additionally, agreement on forceps-impacted sections for the BP-Tr model was high across the BP-Ev and BC-Ev sets (κ = 0.84), confirming its performance is minimally impacted by artifact frames. The BC-Tr model also showed high agreement (κ = 0.86), demonstrating robustness when encountering artifacts it was not trained on. Finally, a direct comparison of the BP-Tr and BC-Tr models on the BP-Ev set yielded strong consistency (κ = 0.81), confirming the minimal influence of the training strategy. Conclusion The near-identical performance and high prediction agreement across all conditions confirm UC-SCALE’s strong robustness to biopsy-related artifacts. Given that cleaning the training data offered no significant advantage, we conclude that an explicit artifact-removal pre-processing step is an unnecessary complexity. References: 1. Gutierrez-Becker B, Fraessle S, Yao H, et al. Ulcerative Colitis Severity Classification and Localized Extent (UC-SCALE): An Artificial Intelligence Scoring System for a Spatial Assessment of Disease Severity in Ulcerative Colitis. J Crohns Colitis. 2025;19(1):jjae187. doi:10.1093/ecco-jcc/jjae187 2. Sandborn WJ, Vermeire S, Tyrrell H, et al. Etrolizumab for the Treatment of Ulcerative Colitis and Crohn’s Disease: An Overview of the Phase 3 Clinical Program. Adv Ther. 2020;37(7):3417-3431. doi:10.1007/s12325-020-01366-2 Conflict of interest: Dr. Gomariz, Alvaro: Employee of F. Hoffmann-La Roche Ltd. Gutierrez Becker, Benjamin: Employee of F. Hoffmann-La Roche Ltd. Girycki, Rafal: Employee of Future Processing S.A. working for F. Hoffmann-La Roche Ltd. Szostak, Przemyslaw: Employee of F. Hoffmann-La Roche Ltd. Stimpel, Bernhard: Employee of F. Hoffmann-La Roche Ltd. Levitte, Steven: Employee at Genentech Inc. Abbasi Sureshjani, Samaneh: Employee of F. Hoffmann-La Roche Ltd.