Systematic Reward Function Design for Reinforcement Learning-Based Continuous Control

Reward formulation plays a pivotal role in reinforcement learning (RL)-based control by determining how competing control objectives are represented during policy learning. However, reward functions are often selected heuristically, and the effects of their mathematical structures on closed-loop behaviour remain insufficiently characterised. This study introduces a systematic reward-design framework that isolates the effects of tracking-error and control-variation penalties through a controlled comparison of ten reward formulations. Quadratic, absolute, and mixed quadratic–absolute tracking penalties are combined with squared and absolute control-variation penalties and evaluated using soft actor–critic (SAC) on two dynamically distinct systems: a DC–DC boost converter and a separately excited DC motor. The results reveal clear trade-offs between tracking accuracy, transient performance, and control-action smoothness. A formulation combining mixed quadratic–absolute tracking penalties with squared control-variation regularisation provides the most favourable overall compromise in both systems. For the DC motor, this formulation achieves only 0.76% higher integral absolute error than the minimum-error formulation while reducing overshoot by approximately 35.9%. The findings demonstrate that reward structure is an important control-design variable and provide practical guidance for balancing tracking performance and control-action smoothness in RL-based continuous control.