P0492Deep learning for automated and objective in vivo assessment of intestinal epithelial barrier in IBD using probe-based confocal laser endomicroscopy

Abstract Background Probe-based confocal laser endomicroscopy (pCLE) allows real-time in vivo assessment of mucosal intestinal barrier structure and function. Non-standardized image interpretation is one of the key limiting factors for pCLE clinical implementation. We developed a suite of deep learning classification models for the automated analysis of pCLE images and videos to objectively quantify intestinal epithelial barrier integrity. Methods A ResNet50-based Convolutional Neural Network (CNN), Model 1, was trained to pre-filter frames for informativeness. For leakage assessment, a dual-branch ResNet50- and ConvNeXtTiny-based CNN, Model 2, was trained to classify informative frames as with epithelial barrier leakage or healing. Separate instantiations of Model 1 and Model 2 were developed and optimized for colonic and ileal datasets, maintaining the core architectural principles across anatomical domains. The video pipeline first processed data sequentially through Model 1 and Model 2. Subsequently, an attention-weighted average of their outputs generated a final video leakage score and classification as leakage or healing. Two pCLE experts (DN and TR) established the ground truth by providing image and video labels. Results The models were trained and tested on pCLE images and videos from IBD patients who underwent colonoscopy with concurrent pCLE at the Ludwig Demling Endoscopy Center of Excellence between January 2017 and December 2019 (training cohort) and between January 2023 and August 2025 (test cohort). Model 1 demonstrated exceptional performance (99% accuracy) across both colonic and ileal datasets. For epithelial barrier leakage or healing assessment, Model 2 exhibited strong performance, achieving a test accuracy of 90% and 91% for colonic and ileal images, respectively. Gradient-weighted Class Activation Mapping (Grad-CAM) visually confirmed that the models’ classifications were based on epithelial features (Figure 1). In the video analysis, the pipeline demonstrated strong performance for both colon and ileum in discriminating between healing and leakage cases with an Area Under the Curve of 0.94 and 0.87 in the discovery cohort and of 0.89 and 0.87 in the test cohort (Figure 2). Conclusion Our deep learning framework provides an automated, objective tool to quantify in vivo epithelial barrier integrity in IBD. The models’ validated robustness on unseen data, with Grad-CAM confirmed epithelial focus, supports their clinical potential for quantifying barrier dysfunction in IBD using pCLE. Conflict of interest: Dr. Noviello, Daniele: Research grant from Pfizer Consulting fees from Abbvie and Dr. Falk Lecturer fees from Ferring and Alfasigma Atreya, Raja: RA has served as a speaker, or consultant, or received research grants from AbbVie, Abivax, AlfaSigma, Arena Pharmaceuticals, Astra-Zeneca, Biogen, Boehringer Ingelheim, Bristol-Myers Squibb, Celgene, Celltrion Healthcare, Dr Falk Pharma, Galapagos, Gilead, GlaxoSmithKline, InDex Pharmaceuticals, Johnson & Johnson, Lilly, Materia Prima, Merck Sharpe & Dohme, Pfizer, Roche Pharma, Takeda Pharma, Viatris. Caprioli, Flavio: Flavio Caprioli served as consultant to: Abbvie, Amgen, MSD, Takeda, Janssen, Roche, Celgene, Bristol-Meyers Squibb, Galapagos, Gllead, Pfizer, Mundipharma, Biogen, Ferring, Eli-Lilly, Nestlè, Lionhealth, AlfaSigma, Dr Falk, Celltrion, Abivax. He received lecture fees from Abbvie, Ferring, Takeda, Allergy Therapeutics, Janssen, Pfizer, Biogen, Sandoz, Tillotts Pharma, Vifor Pharma, AlfaSigma, Celltrion, Eli-Lilly. and unrestricted research grants from Giuliani, Sofar, MSD, Takeda, Abbvie, Celltrion, Pfizer, Actial. Vecchi, Maurizio: M.a.V. has served as speaker for Abbvie, AGPharma, Alfasigma, Biogen, Celltrion, Galapagos, Johnson&Johnson, Eli Lilly, Ferring, Malesci, Pfizer, Sofar, Takeda has served as consultant for Abbvie, Biogen, Bristol-Myers Squibb, Celltrion, Eli Lilly, Giuliani, Johnson&Johnson, Pfizer, Sofar, and Takeda received research support from Giuliani, Pfizer, Sofar, and Takeda. Neurath, Markus Friedrich:. Tontini, Gian Eugenio: Research grant from Pfizer Consulting fees from Abbvie and Dr. Falk Lecturer fees from Ferring and Alfasigma Barbone, Marco: No conflict of interest Waldner, Maximilian: Personal Fees: Takeda Pharma Vertrieb GmbH & Co.KG, Roche Deutschland Holding GmbH, AbbVie Deutschland GmbH & Co.KG Rath, Timo: Research grant from Pfizer Consulting fees from Abbvie and Dr. Falk Lecturer fees from Ferring and Alfasigma