A deep multimodal learning perspective for railway structural health monitoring: A comprehensive review

The increasing deployment of data-driven railway structural health monitoring (SHM) has generated vast amounts of multimodal data. However, the effective integration of these heterogeneous data sources for multimodal learning remains blank, both academically and practically. In view of that, a comprehensive review of deep multimodal learning for railway SHM is conducted, systematically examining the theoretical foundations, data modalities, application scenarios, implementation challenges, and future research directions in this promising field. This article implies two novelities that differs from other related reviews: 1) proposing categorization of railway SHM multimodal data to clarify the modality characteristics and primary purposes of vibration/acoustic, visual, and operational data; 2) discussing recent applications covering rail systems, bridges and tunnels, vehicle components, and environment and electrical equipment, detailing typical modality combinations, model architectures, and downstream tasks. The review reveals a clear evolution of algorithms and tasks in deep multimodal learning methodologies, bridging the gap between methodologies and railway engineering practice. As a consequence, this work provides a foundational reference for researchers and practitioners, paving the way for predictive, intelligent asset management in railway infrastructure.