Spatiotemporal graph-based surrogate modeling and deep reinforcement learning for multi-layer injection-production decision in waterflood reservoirs

This is the authors' abstract. We don't add key points for this paper.

As waterflood reservoirs enter the middle and late development stages, intensified injection-production conflicts, complex remaining-oil distributions, and strong interlayer heterogeneity pose significant challenges to efficient reservoir management. To address these issues, this study developed a graph-based spatiotemporal surrogate modeling framework that integrates a graph convolutional network (GCN), a transformer module, and a long short-term memory (LSTM) network to capture spatial connectivity, temporal dependencies, and nonlinear dynamic responses in multi-layer injection-production systems. The surrogate model was embedded into a deep reinforcement learning framework based on the twin delayed deep deterministic policy gradient (TD3) algorithm for adaptive continuous injection-production control. The proposed method was applied to the Jinzhou South waterflood reservoir using multi-source numerical simulation data. On the test set, the surrogate model achieved high predictive accuracy, with coefficient of determination ( R 2 ) values greater than 0.99 and an average relative error of 3.41% for well-layer production prediction. Under the optimized controls, surrogate-predicted production performance was compared with high-fidelity simulator results for reliability verification. The average relative errors were 2.15% for block-scale production rates and 1.31% for cumulative production, while well-layer predictions maintained R 2 values greater than 0.99. Simulator-based re-evaluation showed that cumulative oil production increased from 955,877.34 m 3 to 964,079.87 m 3 , while cumulative water cut decreased from 22.10% to 21.44%. Compared with GA and DDPG, the proposed framework achieved better oil enhancement and water control, with an 8.00× speed-up over simulator optimization. Overall, the proposed framework provides an accurate and efficient solution for intelligent layered injection-production optimization.