Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE

Katarzyna Maciejowska, Arkadiusz Lipiecki, Bartosz Uniejewski

Abstract

Electricity price forecasts are typically evaluated using accuracy measures such as RMSE and MAE, although these metrics often fail to reflect their economic value in operational decisions. This paper investigates which statistical properties of electricity price forecasts are most relevant for economic performance, using battery energy storage system (BESS) arbitrage as an application. We assess prediction quality along four dimensions: forecast accuracy, intraday error dispersion, association between predicted and realized prices, and the ability to identify daily price extrema. We construct a comprehensive pool of 192 hourly day-ahead electricity price forecasts and use it to evaluate the relationship between proposed quality measures and profits generated for two representative BESS configurations. The results show that traditional accuracy metrics are only weakly correlated with BESS income. At the same time, dispersion- and association-based measures better capture a forecast's economic value by reflecting its ability to reproduce daily price patterns. These findings demonstrate that incorporating complementary evaluation criteria may improve forecast selection and enhance the economic performance of BESS.

Statistical and economic evaluation of forecasts in electricity markets: beyond RMSE and MAE

Abstract

Electricity price forecasts are typically evaluated using accuracy measures such as RMSE and MAE, although these metrics often fail to reflect their economic value in operational decisions. This paper investigates which statistical properties of electricity price forecasts are most relevant for economic performance, using battery energy storage system (BESS) arbitrage as an application. We assess prediction quality along four dimensions: forecast accuracy, intraday error dispersion, association between predicted and realized prices, and the ability to identify daily price extrema. We construct a comprehensive pool of 192 hourly day-ahead electricity price forecasts and use it to evaluate the relationship between proposed quality measures and profits generated for two representative BESS configurations. The results show that traditional accuracy metrics are only weakly correlated with BESS income. At the same time, dispersion- and association-based measures better capture a forecast's economic value by reflecting its ability to reproduce daily price patterns. These findings demonstrate that incorporating complementary evaluation criteria may improve forecast selection and enhance the economic performance of BESS.

Paper Structure

This paper contains 30 sections, 13 equations, 4 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Data
  3. Forecasting electricity prices
  4. ARX model
  5. Direct model of hourly prices
  6. Model of deviation from daily mean
  7. NARX
  8. LEAR model
  9. Model specifications and forecast averaging
  10. Size of a calibration window
  11. VST data transformation
  12. Heterogeneous vs. pooled estimator
  13. Forecast averaging
  14. Summary
  15. Forecast evaluation
  16. ...and 15 more sections

Figures (4)

  • Figure 1: EPEX SPOT hourly day-ahead electricity prices (top), hourly day-ahead system load forecasts (upper middle), renewable generation from solar and wind sources (lower middle), and commodity prices are shown for the period from 2.1.2016 to 31.12.2024. The vertical dashed line marks the end of the 1460-day calibration window for the forecasting models and the beginning of the 2199-day out-of-sample test period.
  • Figure 2: Examples of DA price forecasts. Setup 1 (left panel): forecasts with low dispersion and high association, leading to $\pi^{(1)}$ = 24.68 EUR profit. Setup 2 (right panel): forecasts with high dispersion and low association, leading to $\pi^{(2)}$ = 0.16 EUR profit. Both predictions are characterized by the same RMSE and MAE values.
  • Figure 3: Scatter plots of average daily profits for the BESS-a energy storage calculated for the entire testing set with respect to different error measures.
  • Figure 4: Spearman correlation between forecast quality measures and profits for the BESS-a (top panel) and BESS-b (bottom panel) energy storage. The grey line marks the average daily price of electricity.