Endoscopic diagnosis of eosinophilic esophagitis using a multi-task U-Net: A pilot study

Abstract Background and Aims: Endoscopically identifying eosinophilic esophagitis (EoE) is difficult because of its rare incidence and subtle morphology. We aim to develop a robust and accurate convolutional neural network (CNN) model for EoE identification and classification in endoscopic images. Methods: We collected 548 endoscopic images from 81 patients with EoE and 297 images from 37 normal patients. These datasets were labeled according to the four endoscopic reference score (EREFS) features: edema, ring, exudates and furrow. A multi-task U-Net with auxiliary classifier on various level of skip connections (scaU-Net) was proposed. Then scaU-Net was compared with those of VGG19, ResNet50, EfficientNet-B3, and a typical multi-task U-Net CNNs. The performances of each model were evaluated quantitatively and qualitatively based on accuracy (ACC), area under the receiver operating characteristics (AUROC), and gradient weighted class activation map (Grad-CAM); and also compared to those of 25 human endoscopists. Results: Our sca4U-Net with 4th level skip connection showed the best performances in ACC (86.9%), AUROC (0.93) and outstanding Grad-CAM results compared to other models, reflecting the importance of utilizing the deepest skip connection. Moreover, the sca4U-Net showed generally better performance when compared with endoscopists with various levels of experiences. Conclusions: Our method showed robust performance compared to expert endoscopists, and could assist endoscopists of all experience levels in the early detection of EoE- a rare, but clinically important condition.