Deep Reinforcement Learning for Smart Grid Operations: Algorithms, Applications, and Prospects

With the increasing penetration of renewable energy and flexible loads in smart grids, a more complicated power system with high uncertainty is gradually formed, which brings about great challenges to smart grid operations. Traditional optimization methods usually require accurate mathematical models and parameters and cannot deal well with the growing complexity and uncertainty. Fortunately, the widespread popularity of advanced meters makes it possible for smart grid to collect massive data, which offers opportunities for data-driven artificial intelligence methods to address the optimal operation and control issues. Therein, deep reinforcement learning (DRL) has attracted extensive attention for its excellent performance in operation problems with high uncertainty. To this end, this article presents a comprehensive literature survey on DRL and its applications in smart grid operations. First, a detailed overview of DRL, from fundamental concepts to advanced models, is conducted in this article. Afterward, we review various DRL techniques as well as their extensions developed to cope with emerging issues in the smart grid, including optimal dispatch, operational control, electricity market, and other emerging areas. In addition, an application-oriented survey of DRL in smart grid is presented to identify difficulties for future research. Finally, essential challenges, potential solutions, and future research directions concerning the DRL applications in smart grid are also discussed.