Scalable approach to create annotated disaster image database supporting AI-driven damage assessment

Abstract As coastal populations surge, the devastation caused by hurricanes becomes more catastrophic. Understanding the extent of the damage is essential as this knowledge helps shape our plans and decisions to reduce the effects of hurricanes. While community and property-level damage post-hurricane damage assessments are common, evaluations at the building component level, such as roofs, windows, and walls, are rarely conducted. This scarcity is attributed to the challenges inherent in automating precise object detections. Moreover, a significant disconnection exists between manual damage assessments, typically logged-in spreadsheets, and images of the damaged buildings. Extracting historical damage insights from these datasets becomes arduous without a digital linkage. This study introduces an innovative workflow anchored in state-of-the-art deep learning models to address these gaps. The methodology offers enhanced image annotation capabilities by leveraging large-scale pre-trained instance segmentation models and accurate damaged building component segmentation from transformer-based fine-tuning detection models. Coupled with a novel data repository structure, this study merges the segmentation mask of hurricane-affected components with manual damage assessment data, heralding a transformative approach to hurricane-induced building damage assessments and visualization.