Post-earthquake structural damage detection with tunable semi-synthetic image generation

In the aftermath of an earthquake, conducting rapid structural safety assessments is essential. A Deep Learning-based damage detector capable of automatically analyzing videos from Unmanned Aircraft Systems (UAS) surveys would be highly beneficial for this purpose. Despite significant advancements in object detection using Deep Convolutional Neural Networks (DCNNs), developing an effective post-earthquake damage detector remains challenging due to the scarcity of large, annotated image datasets. In this work, we present a system to create a large number of images where artificial damage instances are applied to real-world three-dimensional (3D) models of buildings and bridges. We defined such images as semi-synthetic. The proposed method relies on the definition, made by human experts, of meta-annotations from which a variety of damage instances can be generated in a controlled way. Semi-synthetic images are designed to augment real-world datasets, enhancing the training process of a DCNN-based damage detector. This semi-synthetic image augmentation can be iteratively refined to target the most critical cases. Experiments conducted on the ‘Image Database for Earthquake damage Annotation’ (IDEA) dataset shown that a detector trained on a combination of real and semi-synthetic images performs better than one trained on real images alone. A damage detector trained using the proposed strategy was then incorporated into a system that analyzes and tracks multiple damage instances in UAS-acquired videos, generating concise summaries of the findings. The effectiveness of the system was validated by the analysis of post-earthquake UAS videos and the production of reports that were reviewed by structural engineering experts. • Multi-class structural damage detection in the aftermath of an earthquake. • A new method to generate semi-synthetic images of damaged buildings and bridges. • Generated images augment real datasets during the training of a damage detector. • A damage tracking and reporting system to analyze videos acquired by drones.