Segmentation of Bridge Components from Various Real Scene Inspection Images

The traditional bridge inspection method, which relies on manual visual inspection, is time-consuming, labor-intensive, and could be dangerous. Recent automated bridge inspection approaches aim to utilize unmanned aerial vehicles (UAVs) and computer vision techniques to collect and analyze images to improve the inspection process. A survey of existing literature and tools shows that defect detection/segmentation has been studied extensively. However, there has been little effort focused on segmenting and characterizing the bridge components that have the defects. The identification and characterization of bridge components is essential for bridge inspection, which can contextualize the defects to determine their importance in maintenance decision making. Moreover, existing bridge component recognition approaches lack generalizability in the presence of a variety of bridge types, complex background scenes, and varying shot sizes. To address these gaps, this paper proposes a convolutional neural network (CNN)-based image segmentation method to segment bridge components, which leverages DeepLabv3+ and pre-training from ImageNet to improve feature extraction and generalizability. The proposed method was trained and tested end to end on 13 classes based on the Federal Highway Administration (FHWA)'s Bridge Inspector's Reference Manual. It achieved a mean precision, recall, F-1 measure, and Intersection over Union (IoU) of 86.7%, 78.2%, 81.4%, and 70.4%, respectively.