In this paper, a series of innovative methods are put forward: in the aspect of dimensionality reduction and feature extraction of high-dimensional data, 3D separable convolution is used to extract the spatial and temporal features of video, 1D Dilated CNN is used to extract the characteristics of sensor time series signals, and Graph convolution network (GCN) is used to model the spatial interaction of trajectory data, In the aspect of multi-modal dynamic spatio-temporal fusion, a dynamic cross-modal spatio-temporal attention mechanism is proposed, which can extract key spatio-temporal features from multiple modes and fuse them adaptively according to correlation, effectively capturing the complex spatio-temporal dependence between modes, In the aspect of online adaptive pattern recognition, an incremental spatio-temporal memory network is designed. By constructing a dynamically expandable memory base, the rapid recognition and classification decision of new patterns are realized, so that the system can dynamically adapt to environmental changes. The experimental results show that the dynamic multi-modal spatio-temporal sensing network proposed in this paper is significantly superior to the existing methods in pattern recognition performance, online adaptability and real-time performance in industrial equipment monitoring and smart city security, which provides a strong technical support for the digital transformation of intelligent monitoring technology in smart cities, industry 4.0 and other fields.
Sun et al. (Sun,) studied this question.