This research proposes a lightweight and adaptable control strategy for managing smart residential photovoltaic–battery systems, aiming to reduce loads’ uncertainty from the distribution grid’s perspective. The idea is to minimize deviations from a scheduled power profile at the meter level. The profile can be defined dynamically, thus creating a highly versatile yet simple to define management system, able to fulfill requests from the end-users as well as the grid operator, or other stakeholders from the energy system, like aggregators. Unlike traditional predictive energy management systems that rely on centralized optimization and continuous communication (like demand response), the proposed approach operates autonomously and is implemented directly at the converter level, enabling high-resolution real-time control, with no or minimal communication with a central unit or the grid operator. The controller is based on a proximal policy optimization agent that processes sequential inputs and historical data to inform its decisions. Several neural network architectures and input configurations are tested, some integrating forecasts with different horizons and accuracy levels. The controllers’ performance is assessed using daily deviation metrics that quantify both the overall error with commitments and the avoidable portion of the deviation relative to the scheduled profile. In addition, this work is validated in real-time on an OPAL-RT device, demonstrating its feasibility for deployment on embedded converter hardware. The proposed method is benchmarked against reference controls: rule-based (for explicability), optimization-based (for theoretical upper bound), and backcasting, an optimization-based control obtained with a delay of a day. Results demonstrate the capability of the reinforcement learning controller to track high-resolution schedules robustly while operating with minimal or no communication for a central unit and at a lower control level than conventional management systems architectures for smart buildings.
Sanchez et al. (Sun,) studied this question.