Structural Vibration Displacement Measurement Using Point Tracking Based on Prior Physical Assumptions

To address tracking difficulties in weak-texture regions and the limited robustness of traditional vision-based structural vibration measurement methods to visual interference, this paper proposes an approach combining prior physical assumptions with deep learning-based point tracking. First, CoTracker3, a dense point tracking model based on the Transformer architecture, is introduced to construct virtual displacement sensors, which effectively overcomes tracking failures caused by weak-texture features in complex environments. Second, based on the physical prior that local regions of a bridge structure can be approximated as rigid bodies, a point tracking mask generation algorithm based on affine consistency is proposed to identify unreliable trajectories caused by visual interference. Finally, leveraging the physical low-rank property of the vibration observation matrix in structural dynamics, an alternating least squares method with mask constraints is employed for matrix completion, achieving reconstruction of the displacement data affected by visual interference. Free-vibration experiments on a cable-stayed bridge model demonstrate that the proposed method can identify structural displacements with high precision under both the normal condition and four simulated visual-interference conditions lasting 5 s, including Gaussian-noise occlusion, vehicle occlusion, reflection interference, and severe shadows. In addition, systematic comparisons with representative vision-based methods are conducted under the normal condition and multiple visual-interference conditions. These comparisons quantitatively evaluate displacement accuracy, grid-point consistency, and data-recovery performance, thereby demonstrating the comprehensive performance of the proposed framework.