The intactness of structures is essential for the functioning of modern society. Structural health monitoring (SHM) is a field of civil engineering focused on the inspection of civil structures to ensure their safe and enduring operation. Digital technologies can support the complex and expensive process of structural inspection, including data capture by unmanned aircraft systems (UAS). Beyond photogrammetric 3D reconstruction, image data can be used for damage detection, with cracks serving as a particularly insightful indicator of structural integrity. Currently, the most powerful approaches to image recognition are artificial neural networks (ANNs). ANNs consist of layers, where each layer processes data based on learned features. Input data is propagated through these layers in a cascaded manner, transforming it into a corresponding output representation. For crack detection, a suitable output representation is a semantic map that indicates the presence and location of cracks. ANNs are data-driven methods that heavily rely on data. For the dominant paradigm of supervised learning, each sample in the dataset must include a corresponding ground truth. Generative approaches to creating such datasets for crack segmentation currently lack the variance and prompt adherence required and typically do not include ground truth labels, rendering manual image labeling necessary. As a result, crack segmentation relies on available datasets, of which numerous smaller and larger ones exist. For the specific case of UAS-based damage detection, two datasets, S2DS and UAV75, are introduced in this thesis. All available datasets differ in terms of surface materials, data formats, acquisition conditions, image quality, and labeling styles. Some datasets reuse images from other datasets. A collection of 20 useful datasets for crack segmentation is integrated into the OmniCrack30k dataset, enabling collective training and effective benchmarking. Benchmarking critically depends on the metrics used. While the standard intersection-over-union (IoU) metric is suitable for blob-like objects, it is less effective for crack segmentation, as it favors wider cracks over narrower ones. To better align with the needs of structural inspection, the centerline intersection-over-union (clIoU) metric is proposed. It emphasizes the detection of the presence of a crack rather than its accurate width and incorporates a tolerance zone of 4 px to account for slight offsets in annotation or prediction. For OmniCrack30k, no new model architecture is developed. Instead, domain adaptation is used to explore the potential of general-purpose approaches for crack segmentation. Point-prompting methods, such as SAM2, show insufficient performance for cracks but may be effective for blob-like damage, such as exposed reinforcement bars. However, when fine-tuned on OmniCrack30k, four popular general-purpose models demonstrate promising performance compared to domain-specific models. Among these, the nnU-Net approach surpasses all other models, including Mask2Former, by a notable margin. Rather than a model, nnU-Net acts as a framework that adapts a U-Net template architecture and training parameters to the dataset through self-configuration. In-depth analyses of nnU-Net's performance on OmniCrack30k reveal that most features of the framework, such as ensembling and test-time augmentation, are effective. Failure cases show that factors such as image blur, low crack resolution, distracting crack-like artifacts, and labeling inconsistencies negatively impact the performance. Experiments demonstrate that the labeling style of the TopoDS subset notably reduces nnU-Net's performance. Removing this dataset from training led to the development of nnU-CrackNet, which achieves a 3 percentage point performance improvement, with clIoU values of 69.4% on the validation and 67% on the test set. nnU-CrackNet performs reasonably well for materials such as asphalt, concrete, and masonry. 'In the wild' conditions, however, remain a particular challenge. While image-level crack segmentation supports structural inspection, localizing cracks within the 3D model of a structure helps assess the criticality of a crack. In real-world scenarios, fine cracks usually do not leave a 3D footprint and manifest only as a radiometric signal in texture space. Introducing the real-world dataset CrackStructures and the semi-synthetic dataset CrackEnsembles, this thesis proposes Enstrect, a modular, stage-based approach to 2.5D crack segmentation. Enstrect uses image-level segmentation models to detect cracks in 2D, maps the semantic information onto a point cloud, and transforms it into individual damage instances. Damage instances are essential for further analyses, such as determining damage extent. The integration of image-level probabilities into a segmented point cloud can be most effectively accomplished through a fusion scheme that combines both the distance to the structural surface and the angular incidence of each view. With nnU-CrackNet as the segmentation model, Enstrect successfully detects surface cracks in practically relevant use cases. Very narrow and very wide cracks, as well as distracting patterns in the surface texture, can reduce Enstrect's performance in other cases. Conclusive analyses show that the performance of crack segmentation strongly depends on image quality, with image blur and noise having the strongest impact. Another key determinant of detection performance is the width of the crack. A comfort zone for nnU-CrackNet is identified for cracks with widths between 1.3 and 24 px. While minor changes in this comfort zone are observed under conditions of image blur, reduced contrast between the crack and the background narrows the zone to 2 to 16 px. Reconsideration of the sampling theorem provides a plausible explanation for the detectability of cracks below the Nyquist rate. Runtime experiments reveal that for a collection of 1,000 images, each with 67 MP, nnU-CrackNet requires processing times on the order of days.
Christian Benz (Tue,) studied this question.