Rapid geological mapping based on UAV imagery and deep learning texture classification and segmentation

Geological mapping in dynamic coastal areas is crucial, but traditional methods are laborious and expensive. Employing uncrewed aerial vehicle (UAV) - based mapping for high-resolution imagery, combined with deep learning for texture classification and segmentation, offers a promising improvement. In the AI-aided Assessment of Mass Movement Potentials Along the Steep Coast of Mecklenburg Western Pomerania project, we explore the use of deep learning for geological mapping. We conduct repetitive UAV surveys across five distinct coastal areas, documenting various cliff types under different lighting and seasonal conditions. The imagery yields texture patterns for categories such as vegetation, chalk, glacial till, sand, water and cobble. We apply two strategies: classification and semantic segmentation. Classification predicts one label per texture patch, while semantic segmentation labels each pixel. Classification requires distinct files with pre-labeled textures, whereas segmentation needs a training dataset with label masks, assigning class values to each texture pixel. We employ Convolutional Neural Networks (CNN) for classification tasks, designing custom nets with convolutional blocks and attention layers, and testing existing architectures like ResNet50. We evaluate classification performance using accuracy measures and run sensitivity analysis to identify the smallest effective patch size for texture recognition. The effective patch size determines the final mapped class resolution. Classification is less detailed than segmentation but potentially more generalizable. For semantic segmentation, we employ UNet architectures with encoder-decoder structures and attention gates for improved image context interpretation. We evaluate segmentation using the intersect over union index (IoU). Due to the need for extensive, accurate training data, we employ data augmentation to create artificial datasets blending real-world textures inspired by the Prague texture dataset. Classification results show about 95% accuracy across target classes using RGB image input. Notably, the pre-trained ResNet50 exhibits moderate performance in texture recognition and is outperformed by simpler net designs trained from scratch. However, it shows adequate performance when pre-trained weights are neglected. For overall classification improvement, we anticipate that adding a Near Infrared band (NIR) will enhance classification, particularly for vegetation and glacial till, which are currently prone to misclassification. Semantic segmentation yields IoUs of around 0.94 on artificial datasets. However, when applied to real-world imagery, the models show a noisy performance, yielding significant misclassifications. Thus, better generalization requires further fine-tuning and possibly integrating real-world data along with artificial datasets. Also, further experiments with data augmentation by extending the dataset and introducing different complexity levels could provide better generalization to real-world data. In summary, combining UAV mapping with AI techniques holds significant potential for improving geological mapping efficiency and accuracy in dynamic coastal environments, providing reliable parametrization for data-driven models that require up-to-date geological information in high resolution.