Gas-Lift Optimization Using Physics-Informed Deep Reinforcement Learning

Real-time optimization (RTO) methodologies have become essential for optimal process operation in the oil and gas industries. Typically, RTO is based on a steady-state model (steady-state real-time optimization─SSRTO) and operates as a closed-loop optimizer. However, this technique can result in suboptimal policies due to steady-state waiting and plant-model mismatch issues. To alleviate these problems, we combine this closed-loop optimizer with a data-driven residual optimizer based on deep reinforcement learning (DRL). The idea is to use the SSRTO solution's stability and convergence properties while reducing plant-model mismatch through a residual DRL controller that can tune the optimal inputs calculated by the SSRTO based on transient measurements (i.e., real data). Our proposed methodology delivered robust online control policies compared to dynamic real-time optimization (DRTO) for closed-loop control, even when the plant-model mismatch was simulated by introducing parameter uncertainty and noise in the plant and using an SSRTO model with structural uncertainty. As a result, its economic gain was improved compared to the SSRTO solution, with an average computational cost 11 times lower than DRTO, making it suitable for online applications.