Inductive biased-deep reinforcement learning methods for flow control: Group-invariant and positional-encoding networks improve learning reproducibility and quality

Flow control is key to maximize energy efficiency in a wide range of applications. Despite the advancements in deep reinforcement learning (DRL) technology, there are practical difficulties due to computational cost of exploration. Inductive biases, corresponding to prior assumptions incorporated into a deep learning architecture, can help implicitly satisfy given physical laws and thereby improve training efficiency. In this study, we propose an inductive biased-DRL method for effective flow control by developing a DRL framework coupled with a group-invariant network and positional encoding. Our methods leverage multi-agent approach to exploit policy invariance in space, in combination with developed group-invariant networks to ensure local symmetry invariance. Additionally, a positional encoding is incorporated to provide location information to the agents, mitigating action constraints from strict invariance. The proposed methods are verified using a case study of Rayleigh−Bénard convection−Nusselt number control. The group-invariant neural networks (GI-NNs) show faster convergence compared to the fully connected layer, achieving better average policy performance. The GI-NNs not only cut training time in half but also notably enhance learning reproducibility. Interestingly, positional encoding specializes in improving learning quality. These results demonstrate that developing a suitable inductive-based DRL architecture by fluid mechanics knowledge of each control problem is essential. We believe that the results of this study will not only inspire novel inductive biased-DRL methods with invariant and unique representations but also provide useful insights for flow control applications.