3D vision-based out-of-plane displacement quantification for steel plate structures using structure-from-motion, deep learning, and point-cloud processing

In this paper, a novel accurate and economical 3D computer vision-based framework is proposed to quantify out-of-plane displacements of steel plate structures. First, a sequence of image frames of the steel plate structures of interest is collected. Second, using image association, structure-from-motion, and multi-view stereo algorithms, a 3D point cloud of the steel plate structures and their surroundings is created. Third, an efficient 3D object detection method based on convolutional neural networks is developed and implemented to identify the steel plate structures in the 3D point cloud. Last, the out-of-plane displacements of the steel plate structures are quantified using point cloud postprocessing algorithms. The proposed framework has been implemented on a steel plate damper and a full-scale steel corrugated plate wall panel, which are commonly used in structural and earthquake engineering applications. The results indicate the developed framework can successfully localize the steel plate components in the 3D scene and accurately quantify the out-of-plane structural displacements with an average accuracy of ∼1 mm. The implementation shows the proposed framework can accurately and efficiently quantify the out-of-plane displacements of steel plate structures in realistic engineering applications.