Deep Learning Sewer Defects Detection: Comparison of Multilabel Utility-Specific Classification ModelsBased on Convolutional Neural Networks

Closed-circuit television (CCTV) is widely employed for identifying sewer defects. This comparative study develops utility-specific multilabel classification models for three communities in Western Canada (Utilities A-C), based on convolutional neural networks (CNNs), for identifying multiple sewer defects in CCTV video images. The models are trained, validated, and tested with frames extracted from CCTV videos. They are tailored for the different characteristics of the individual utilities and explored to evaluate the benefit of transfer learning. For each utility, we examine three existing CNN backbone architectures, previously trained on the ImageNet data set: MobileNet V3 Large, ResNet50, and ResNet101; two model configurations: one-stage for which all defects and nondefects are classified in one model, and two-stage for which nondefects and defects are sorted first, and then all detected defects are classified; and two testing procedures: using random frames from all videos and using all frames from a subset of the videos set aside for testing purposes and resulting in twelve models per utility. The former testing procedure may be used to evaluate whether the model can perform quality assurance and quality control assessment of previously inspected videos and the latter to identify defects in images from unexamined videos. Performance evaluation of the models reveals that two-stage models result in the highest performance. For models validated using random frames from all videos, the ResNet models outperform MobileNet V3 Large, for which model accuracies for Utilities A, B, and C, are 0.95, 0.70, and 0.98, respectively. Models validated using frames from a subset of videos do not classify different defects well but distinguish between nondefective and defective images with accuracies higher than 0.50, the best being 0.80. Transfer learning applications show that models trained for utilities with larger data sets may be further fine-tuned and tested on data for utilities with smaller data sets, and the resulting models improve on models that were solely fine-tuned and tested using data for the smaller utilities.