Bridging classical and artificial intelligence-based flow control: An expert-guided deep reinforcement learning framework

Deep Reinforcement Learning (DRL) is a powerful paradigm for discovering non-linear control strategies in complex fluid dynamics. However, its application to high-fidelity simulations is often hampered by prohibitive sample inefficiency and the risks of “cold-start” exploration. To overcome these limitations, an Expert-Guided Soft Actor-Critic (EG-SAC) framework is proposed, which synergistically integrates prior knowledge from a classical, particle swarm optimization-optimized proportional-integral-derivative controller into the DRL agent. The framework employs a two-stage learning process, beginning with an offline phase where behavioral cloning initializes the policy and expert demonstrations pre-fill the replay buffer and an online fine-tuning stage where a composite loss function, featuring a decaying expert-regularization term, provides continuous guidance to the agent. This framework is applied to active flow control for a circular cylinder at a Reynolds number (Re) of 100. The results demonstrate a threefold advantage: first, the EG-SAC agent achieves immediate operational safety, exhibiting near-expert performance from the first epoch, while the standard SAC agent initially suffers from detrimental, high-drag explorations; second, the sample efficiency is significantly enhanced, with asymptotic convergence accelerated by ∼19%; finally, the converged EG-SAC policy achieves a drag reduction of 7.93%, surpassing the 5.21% achieved by the best linear controller and demonstrating the ability to discover superior non-linear control laws. The superior performance stems from an enhanced base pressure recovery via a further elongated recirculation bubble. This work presents a robust and efficient methodology for applying DRL to fluid mechanics, bridging the gap between the stability of classical control and the high-performance adaptability of data-driven methods.