Secure CO2 Sequestration: A Deep Reinforcement Learning Perspective on Injection Rate Management

Summary Secure and efficient geological CO2 storage decisions can be supported by models of subsurface porous structure and of the fluid flow in the porous network. Geological and flow modeling is employed in order to optimize the CO2 storage process during the lifetime of the operation. The aim is to minimize the fraction of mobile CO2. To account for injection rates in target wells that are adjusted at different time intervals, a sequential decision-making model has been adopted. The method includes the future evolutions of uncertainties and includes future decisions at each time step ‘t’ until the end of operation. To address the curse of dimensionality associated decision variables (continuous and exponential growth along the time horizon), a Reinforcement Learning (RL) Approach - Proximal Policy Optimization (PPO)- is formulated. The example case of the CO2 injection model includes the 11th Society of Petroleum Engineers Comparative Solution Project, focusing on CO2 storage operations in geological settings of realistic complexity. The optimal policy suggested by the RL model provides a control policy that instantaneously maps the observed information (state) to the policy. The RL model and framework of the implementation are presented and discussed. Moreover, the research discusses the computational efficiency and scalability of the framework, providing insights into its potential feasibility for larger-scale CO2 storage projects. By employing state-of-the-art artificial intelligence methods, this research presents an efficient and realistic model for supporting CO2 injection decisions leading to effective and secure solutions for CO2 storage.