Enhancing the resilience of urban drainage system using deep reinforcement learning

Real-time control (RTC) is an effective method used in urban drainage systems (UDS) for reducing flooding and combined sewer overflows. Recently, RTC based on Deep Reinforcement Learning (DRL) has been proven to have various advantages compared to traditional RTC methods. However, the existing DRL methods solely focus on reducing the total amount of CSO discharge and flooding, ignoring the UDS resilience. Here, we develop new DRL models trained by two new reward functions to enhance the resilience of UDS. These models are tested on a UDS in eastern China, and found to enhance UDS resilience and, simultaneously, reduce the total amount of flooding and CSO discharges. Their performance is influenced by the rainfalls and the DRL types. Specifically, different rainfalls lead to different resilience performance curves and DRL model generalization. The value-based DRL model trained with the duration-weighted reward achieves the best performance in the case study.