Data Compression for Time Series Modelling: A Case Study of Smart Grid Demand Forecasting

TL;DR

This study tackles the challenge of managing large volumes of high-frequency time series data in smart energy systems by evaluating Discrete Wavelet Transform (DWT)-based lossy compression. It benchmarks three forecasting models—Ordinary Least Squares (OLS), XGBoost, and TiDE—on a seawater intake system under varying compression rates and biorthogonal wavelets, using Normalized Mutual Information (NMI) to quantify information retention. Key findings show that XGBoost is robust to compression artifacts across wavelets and rates, while OLS is sensitive to smoother, high-rate compressions; TiDE displays competitive but variable performance. The results demonstrate that, with carefully chosen wavelets and compression levels, wavelet-based compression can dramatically reduce data storage needs without sacrificing forecasting accuracy, informing edge- and cloud-based deployment in smart grids and related high-frequency forecasting domains.

Abstract

Efficient time series forecasting is essential for smart energy systems, enabling accurate predictions of energy demand, renewable resource availability, and grid stability. However, the growing volume of high-frequency data from sensors and IoT devices poses challenges for storage and transmission. This study explores Discrete Wavelet Transform (DWT)-based data compression as a solution to these challenges while ensuring forecasting accuracy. A case study of a seawater supply system in Hirtshals, Denmark, operating under dynamic weather, operational schedules, and seasonal trends, is used for evaluation. Biorthogonal wavelets of varying orders were applied to compress data at different rates. Three forecasting models - Ordinary Least Squares (OLS), XGBoost, and the Time Series Dense Encoder (TiDE) - were tested to assess the impact of compression on forecasting performance. Lossy compression rates up to $r_{\mathrm{lossy}} = 0.999$ were analyzed, with the Normalized Mutual Information (NMI) metric quantifying the relationship between compression and information retention. Results indicate that wavelet-based compression can retain essential features for accurate forecasting when applied carefully. XGBoost proved highly robust to compression artifacts, maintaining stable performance across diverse compression rates. In contrast, OLS demonstrated sensitivity to smooth wavelets and high compression rates, while TiDE showed some variability but remained competitive. This study highlights the potential of wavelet-based compression for scalable, efficient data management in smart energy systems without sacrificing forecasting accuracy. The findings are relevant to other fields requiring high-frequency time series forecasting, including climate modeling, water supply systems, and industrial operations.

Figures (7)

• Figure 1: Dataset sizes for the training, validation and testing datasets.
• Figure 2: Example of model predictions for all 3 time series forecasting models. Each model used a context of six hours to make predictions for another six hours, of which one hour was directly predicted and the remaining was computed autoregressively.
• Figure 3: Example subset of the compressed, normalized target variable. The rows show different compression rates $r \in \{0.9, 0.95, 0.99\}$ while the columns show different families of biorthogonal wavelets.
• Figure 4: RMSE for every model against the compression rate $r$. The rows show different models, OLS, XGBoost and TiDE, while the columns show different families of biorthogonal wavelets. The boxplots show the distribution of RMSE's across the $N_{\mathrm{test}}=12$ different testing datasets, including outliers. The whiskers represent the 1.5 Interquantile Range (IQR).
• Figure 5: Normalized Mutual Information (NMI) for each wavelet as the compression is increased. Note that the x-axis is displayed on a logit-scale.