Pilot Study of Synthetic Data Augmentation for Vision-based Excavator Pose Estimation using Generative Models

With the advancement of computational power and data utilization techniques, significant research has been undertaken on automation technologies for monitoring construction equipment. Particularly, deep learning and computer vision techniques commonly applied for construction monitoring require a massive amount of onsite data (e.g., site images). To address this issue, this paper presents a novel data generation method that allows for synthesizing images and pose data in a simulation environment and then transforming these images into synthetic images in a real excavator style using a CycleGAN (cycle generative adversarial networks)-based generative model. For validation, experiments were carried out to evaluate the photorealism performance and geometry preservation performance of the proposed model. The results showed that the proposed model achieved the successful generation of synthetic images with style and content information as intended. This study thus may serve as the foundation for exploring a new way to obtain site data more easily, which has been one of the biggest issues for deep learning studies in construction.