Optimal Control of Mobile Energy Storage via Knowledge‐Guided Deep Reinforcement Learning

ABSTRACT While mobile battery energy storage systems (MBESSs) are typically used to improve the stability of power systems, their ability to move also creates good opportunities for businesses to earn money through energy arbitrage. This profit depends heavily on decisions about timing and location, and is affected by uncertain conditions like fluctuating electricity prices and traffic. However, finding the best real‐time control strategy that considers long‐term profit and these uncertainties requires significant computing power. To tackle this issue, this paper presents a deep reinforcement learning framework for MBESSs designed to get the most profit from market arbitrage. Within this framework, we introduce the Knowledge‐Assisted Deep Deterministic Policy Gradient (KA‐DDPG) algorithm to learn the best policy more efficiently. The core novelty of KA‐DDPG lies in its probabilistic hybrid action selection mechanism that unifies the agent's learned policy, offline expert criteria, and random exploration to manage the complex hybrid action space. Additionally, a two‐phase guidance strategy is implemented to transition from offline‐based to real‐time‐based criteria actions, ensuring both learning acceleration and policy robustness under computational constraints. Our rigorous statistical evaluations demonstrate that the proposed KA‐DDPG approach leads to a 3%–7% improvement in average profits over the state‐of‐the‐art Soft Actor‐Critic baseline. Furthermore, it achieves exceptional policy stability, exhibiting a variance reduction of over 60% compared to standard DRL baselines and over 92% compared to the deterministic closed‐loop MPC. The KA‐DDPG algorithm also substantially speeds up the learning phase, validating its efficacy for real‐time MBESS control under high uncertainty.