Development, Application and Experimental Validation of Reinforcement Learning-Based Strategies for Process Control

Since the last decade, artificial intelligence (AI) has been rapidly gaining prominence in the field of process control. In this regard, hybrid AI-based methods have proved their dominance over first principle-based and stand-alone AI methods. Among different AI-based methods, reinforcement learning (RL)-based actor-critic (AC) methods have an edge because of their adaptive features, a vital quality for process control where uncertainties and availability of limited process information are fairly common. With this motivation, the overarching objective in this work is to develop fast-acting process control strategies that integrate AC methods with other AI-based and first principle-based methods in order to boost controller performance. To that end, a comprehensive literature survey is conducted initially that summarizes past developments in this domain. A comparative evaluation study is performed, which shows the superiority of deep deterministic policy gradient (DDPG) and twin-delayed DDPG methods over other AC methods, in controlling different in-silico processes with varying nonlinearity. Two novel control strategies are developed by assimilating DDPG with multiple neural network (MNN) to generate an MNNRL controller, and a novel transformed inverse model (TIM) control technique to generate a TIM-RL controller. Both controllers are tested in different in silico case studies with varying degree of nonlinearity and complexity. An experimental set up for the control of a highly nonlinear pH neutralization process is built and utilized to test and validate both controllers. The results show that both MNNRL and TIM-RL can improve controller performance efficiency by 40%, on an average, with 99% reduction in computation time to determine control actions, when compared to the conventional non-linear model predictive control method.