Real-Time Energy Pricing in New Zealand: An Evolving Stream Analysis

TL;DR

The paper presents a real-time, streaming dataset of New Zealand Dispatch Energy Prices from EMI, addressing a gap in datasets suitable for streaming regression, PI, and drift/anomaly analysis. It details preprocessing steps (PoC regionalization, trig-encoded time features, and multiple target horizons) to enable continuous learning, and evaluates multiple streaming algorithms (notably SOKNL) along with AdaPI for prediction intervals and drift/anomaly detectors. Key findings show robust regression performance from SOKNL, effective but nuanced PI coverage with AdaPI, and frequent concept drifts and challenging anomaly detection in the data. The work highlights practical implications for real-time energy price forecasting and outlines future directions, including incorporating meteorological data and expanding to additional regions.

Abstract

This paper introduces a group of novel datasets representing real-time time-series and streaming data of energy prices in New Zealand, sourced from the Electricity Market Information (EMI) website maintained by the New Zealand government. The datasets are intended to address the scarcity of proper datasets for streaming regression learning tasks. We conduct extensive analyses and experiments on these datasets, covering preprocessing techniques, regression tasks, prediction intervals, concept drift detection, and anomaly detection. Our experiments demonstrate the datasets' utility and highlight the challenges and opportunities for future research in energy price forecasting.

Figures (6)

• Figure 1: PoCs Overview
• Figure 2: Showcase of the encoded periodical information, PoC: HAM0331.
• Figure 3: Demonstration of target values as mean and median prices.
• Figure 4: SDN0331 PoC: Prequential R$^2$ Score with 1000 Window Size
• Figure 5: Visualized Example of Prediction Intervals Over Time