Markov-Decision-Process-Assisted Consumer Scheduling in a Networked Smart Grid

Many recently built residential houses and factories are equipped with facilities for converting energy from green sources, such as solar energy, into electricity. Electricity consumers may input the extra electricity that they do not consume into the smart grid for sale, which is allowed by law in countries such as Japan. To reduce peak-time electricity usage, time-varying pricing schemes are usually adopted in smart grids, for both the electricity sold to consumers and the electricity purchased from consumers. Thanks to the development of cyber-physical systems and advanced technologies for communication and computation, current smart grids are typically networked, and it is possible to integrate information such as weather forecasts into such a networked smart grid. Thus, we can predict future levels of electricity generation (e.g., the energy from solar and wind sources, whose generation is predominantly affected by the weather) with high accuracy using this information and historical data. The key problem for consumers then becomes how to schedule their purchases from and sales to the networked smart grid to maximize their benefits by jointly considering the current storage status, time-varying pricing, and future electricity consumption and generation. This problem is non-trivial and is vitally important for improving smart grid utilization and attracting consumer investment in new energy generation systems, among other purposes. In this paper, we target such a networked smart grid system, in which future electricity generation is predicted with reasonable accuracy based on weather forecasts. We schedule consumers' behaviors using a Markov decision process model to optimize the consumers' net benefits. The results of extensive simulations show that the proposed scheme significantly outperforms the baseline competing scheme.