Simulating 4D scenes of rockfall and landslide activity for improved 3D point cloud-based change detection using machine learning

Terrestrial Laser Scanning (TLS) systems have been refined to automatically and continuously scan defined areas with high temporal resolution (sub-hourly), leading to the development of Permanent Laser Scanning (PLS). This temporal resolution requires the development of new methods for efficient extraction of change information. The creation of labeled 4D point clouds (3D+time), classified by surface change type, remains time-consuming. This hinders the evaluation of change detection methods and the training of machine learning (ML) and deep learning (DL) models. This study explores how synthetic 4D point clouds can be effectively utilized for detecting and classifying spatiotemporal changes. We combine simplified process path simulations, simulated PLS, and change detection methods (e.g. M3C2) 1. This combination is used to automatically evaluate calculated distances compared to a pre-defined reference. It also generates labeled 4D training datasets for ML/DL approaches. We adapted the Gravitational Process Path model (GPP) 2 to create gravity-influenced process paths for our PLS simulations. Utilizing these paths, we simulate two different scenarios, 1) including a forest situated on top of a large landslide and 2) an outcrop with rockfall activity. For the forest scenario, a constant velocity is applied to each tree to simulate slope movement. The velocity of the objects in the rockfall scene is determined by the GPP model. Dynamic 3D scenes are generated from these scenarios and used as input for Virtual Laser Scanning (VLS). Realistic simulation of LiDAR surveys (of these virtual scenes) is achieved by using the open-source simulator HELIOS++ 3. This workflow allows for the determination of the accurate position of each object at any given time. It provides reference data that is usually unavailable in real data acquisitions. In the rockfall scenario, M3C2 distances are calculated, and areas of similar change are clustered. For the forest located on the landslide, 2D and 3D displacement vectors are derived from the displacement of the tree trunks. These changes are then compared to the actual change occurring between epochs. Furthermore, the time steps between each epoch can be chosen arbitrarily, enabling the exploration of various scenarios and processes using labeled point clouds at any temporal resolution. Preliminary results suggest that this workflow can assist in determining the scan resolution required to detect changes of a specific size and magnitude. We establish a simulation-based error margin for each method used by comparing the results to the reference data. This enables direct evaluation of method performance during implementation. We demonstrate the potential of combining process simulation and laser scanning simulation for resource efficient planning of TLS and PLS campaigns, geographically sound generation of dynamic point clouds, the evaluation of change detection and quantification methods, and generating labeled point clouds as training data for 4D ML/DL methods. References:1 py4dgeo: https://github.com/3dgeo-heidelberg/py4dgeo 2 Wichmann, V. (2017): https://doi.org/10.5194/gmd-10-3309-2017. 3 Winiwarter, L. et al. (2022): https://doi.org/10.1016/j.rse.2021.112772.