Big data analytics for photovoltaic and electric vehicle management in sustainable grid integration.

This article focuses on developing a big data analytics framework using PySpark, a Python API for Apache Spark, to address challenges in short-term forecasting of photovoltaic (PV) power generation and clustering of electric vehicle (EV) charging load patterns. Utilizing extensive datasets from the Yulara Solar Plant in Australia and Palo Alto EV charging stations in California, the study applies machine learning regression algorithms (linear regression, decision trees, random forests, gradient-boosted trees) for PV power prediction and clustering algorithms (K-means, Gaussian Mixture Models, bisection K-means) for EV user behavior analysis. Results indicate that feature expansion improves forecasting accuracy at the cost of increased training time, while feature extraction offers a balance between accuracy and computational efficiency; K-means clustering outperformed other methods in identifying distinct EV charging patterns across weekdays. The research demonstrates PySpark’s effectiveness in handling high-dimensional, large-scale datasets for enhancing power system management and suggests future integration with real-time grid components and advanced deep learning techniques.