Big data analytics for photovoltaic and electric vehicle management in sustainable grid integration

In recent years, integration of sustainable energy sources integration into power grids has significantly increased data influx, presenting opportunities and challenges for power system management. The intermittent nature of photovoltaic power output, coupled with stochastic charging patterns and high demands of electric vehicles, places considerable strain on system resources. Consequently, short-term forecasting of photovoltaic power output and electric vehicle charging load becomes crucial to ensuring stability and enhancing unit commitment and economic dispatch. The trends of energy transition accumulate vast data through sensors, wireless transmission, network communication, and cloud computing technologies. This paper addresses these challenges through a comprehensive framework focused on big data analytics, employing Apache Spark that is developed. Datasets from Yulara solar park and Palo Alto's electric vehicle charging data have been utilized for this research. The paper focuses on two primary aspects: short-term forecasting of photovoltaic power generation and the exploration of electric vehicle user clustering addressed using artificial intelligence. Leveraging the supervised regression and unsupervised clustering algorithms available within the PySpark library enables the execution of data visualization, analysis, and trend identification methodologies for both photovoltaic power and electric vehicle charging behaviors. The proposed analysis offers significant insights into the resilience and effectiveness of these algorithms, so enabling informed decision-making in the area of power system management.