LLM-Enhanced Feature Engineering for Multi-Factor Electricity Price Predictions

TL;DR

The paper tackles forecasting electricity price volatility in liberalized markets, focusing on the NSW NSW market. It proposes FAEP, a framework that combines LLM-based feature augmentation (via Retrieval-Augmented Generation with external NSW weather data) and a hybrid XGBoost-LSTM model to handle high-dimensional, nonlinear relationships and temporal dependencies. Key contributions include a feature selection mechanism that incorporates volatility indicators like $CV_t$ and $J_t$, dimensionality reduction with Kernel PCA, and a residual-weighting scheme for model fusion; integration of a weather-feature confidence score from LLaMA-Index further enhances predictive accuracy. Experimental results demonstrate state-of-the-art performance with substantial reductions in MAE/MSE and robust ablation findings, underscoring the impact of external weather information and the complementary strengths of XGBoost and LSTM for volatility forecasting in the NSW electricity market.

Abstract

Accurately forecasting electricity price volatility is crucial for effective risk management and decision-making. Traditional forecasting models often fall short in capturing the complex, non-linear dynamics of electricity markets, particularly when external factors like weather conditions and market volatility are involved. These limitations hinder their ability to provide reliable predictions in markets with high volatility, such as the New South Wales (NSW) electricity market. To address these challenges, we introduce FAEP, a Feature-Augmented Electricity Price Prediction framework. FAEP leverages Large Language Models (LLMs) combined with advanced feature engineering to enhance prediction accuracy. By incorporating external features such as weather data and price volatility jumps, and utilizing Retrieval-Augmented Generation (RAG) for effective feature extraction, FAEP overcomes the shortcomings of traditional approaches. A hybrid XGBoost-LSTM model in FAEP further refines these augmented features, resulting in a more robust prediction framework. Experimental results demonstrate that FAEP achieves state-of-art (SOTA) performance compared to other electricity price prediction models in the Australian New South Wale electricity market, showcasing the efficiency of LLM-enhanced feature engineering and hybrid machine learning architectures.

Figures (3)

• Figure 1: The Overview of the LLM Integrated XGBoost-LSTM Model, demonstrating the process of predicting electricity market prices by evaluating and integrating relevant features.
• Figure 2: Forecast Comparison Results
• Figure 3: Diebold–Mariano Test Rejection Rates for the Out-of-Sample Volatility Forecasts.