Chaotic Time Series Prediction Of Beam-Ring Structured Systems Based On The S-Transformer With Convolutional Attention Mechanism

Abstract Since its introduction, the Transformer has demonstrated robust performance. However, its capacity to predict nonlinear dynamical systems remains constrained. In this study, we enhance the structure and propose a novel network model, S-Transformer (Simplified Transformer), which exhibits high efficiency and accuracy in forecasting the chaos time series. Aconvolutional attention mechanism is introduced in place of the attention mechanism inherent to the Transformer architecture, which replaces the traditional multi-head attention mechanism. The Mean Squared Error (MSE) is employed as a metric for evaluating the accuracy of the neural network model's prediction results. The numerical simulations demonstrate that the prediction accuracy of the S-Transformer is 75% higher than the LSTM ED under the same experimental conditions. Furthermore, the prediction accuracy of the S-Transformer with Convolutional Attention Mechanism (Conv-AT) is 96% higher than the S-Transformer. It can be concluded that the enhanced network exhibits a pronounced capability of complex nonlinear systems.