Reward design for deep Reinforcement Learning driven wind farm control: What matters for optimal performance

The reward function is the sole source of feedback in reinforcement learning (RL), yet wind farm flow control studies often treat it as an implementation detail. A review of 24 RL-based wake steering studies reveals fragmentation across reward formulations, no consensus on core design choices, and frequent underreporting. Reported differences may therefore reflect reward design as much as algorithm choice. We treat reward design as the primary research question across a three-phase protocol of over 500 training runs. Phase 1 screens 14 reward formulations under laminar and turbulent inflow using Soft Actor–Critic (SAC) on a 3-turbine inline farm. We observe a 38.7 percentage point performance spread, of which 27 percentage points come from numerical scaling alone, establishing reward formulation as a first-order design choice. Reward rankings then shift under turbulent inflow: laminar champions retain as little as 34% of their gain, while normalized and baseline-relative rewards retain up to 88%. Phase 2 compares the five most robust rewards across SAC and Proximal Policy Optimization (PPO) on 3 × 1 and 3 × 3 layouts: SAC outperforms PPO by 5–10 percentage points, and reward rankings are algorithm-dependent, so reward and algorithm must be selected jointly. Phase 3 penalty sweeps show that moderate travel penalties preserve power gains while cutting cumulative yaw travel by 30–40%, whereas excessive penalties induce a parking paradox in which the controller becomes trapped at the yaw limits. We propose design heuristics and a candidate starting configuration for RL-based wind farm control in the settings studied here.