Neural Networks and Reinforcement Learning in Control of Water Systems

In dynamic real-time control (RTC) of regional water systems, a multicriteria optimization problem has to be solved to determine the optimal control strategy. Nonlinear and/or dynamic programming based on simulation models can be used to find the solution, an approach being used in the Aquarius decision support system (DSS) developed in The Netherlands. However, the computation time required for complex models is often prohibitively long, and therefore such a model cannot be applied in RTC of water systems. In this study, Aquarius DSS is chosen as a reference model for building a controller using machine learning techniques such as artificial neural networks (ANN) and reinforcement learning (RL), where RL is used to decrease the error of the ANN-based component. The model was tested with complex water systems in The Netherlands, and very good results were obtained. The general conclusion is that a controller, which has learned to replicate the optimal control strategy, can be used in RTC operations.