Optimization of Real‐Time Reservoir Operations With Markov Decision Processes

In recognition of hydrologic uncertainty and seasonality, reservoir inflows are described as periodic Markov processes. The optimization of reservoir operations involves determination of the optimal release volumes in the successive time periods so that the expected total rewards resulting from the operations are maximized. A two‐stage optimization framework, which consists of a real time model followed by a steady state model, is proposed. The steady state model that describes the convergent nature of the prospective future operations is regarded as a periodic Markov decision process and is optimized with the generalized policy iteration procedure. This result is in turn used as an interim step for deriving the optimal immediate decisions for the current period in the real‐time model. Significant computational efficiency results from this framework and the respective optimization procedure.