A vision foundation model-based approach for robust and generalizable road marking damage evaluation using unmanned aerial vehicle images

Road markings are an essential component of the modern transportation system. During the road operation period, clear markings are important for the autonomous lane-keeping system and dynamic traffic management. Modern transportation planning and maintenance practices highlight the significance of an accurate program to investigate marking conditions. Recently, computer vision has become prevalent in monitoring transportation infrastructures. However, the real-world variability brings substantial challenges to deep learning programs. Based on Unmanned Aerial Vehicle (UAV) images, we propose Vision Foundation Model (VFM)-based approach for robust marking damage detection and evaluation. First, an image selection method is designed to find representative instances. Self-distillation with No Labels Version Two (DINOv2), Modified Hausdorff Distance, Uniform Manifold Approximation and Projection (UMAP) and K-means clustering are performed to understand the underlying trend within UAV images. Second, we conduct You Only Look Once (YOLO) to localize damages. Third, the semantic segmentation is launched to handle road markings. Without pixel-level annotations, we carry out Segment Anything Model (SAM) to estimate the initial state of markings. Fourth, our program performs a thresholding step to calculate the missing ratio and severity of damaged markings. We test the proposed method in practical applications. The computation results indicate that our VFM-based method exhibits competitive capabilities in training efficiency, prediction accuracy and generalization ability. Based on the image selection program, YOLO significantly improves detection accuracy and mitigates training cost. Our SAM-driven program removes the annotation step and handles image variations. Considering its robustness and generalization, VFM-based method has potential for various future applications.