Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Lightweight Multi-View Approach to Short-Term Load Forecasting

Julien Guité-Vinet, Alexandre Blondin Massé, Éric Beaudry

TL;DR

The paper addresses short-term load forecasting in settings with limited data and the risk of overfitting large models by introducing a lightweight multi-view framework. It combines single-value embeddings, a scaled time-range input of critical lags, and embedding dropout within a transformer-based encoder–decoder to fuse multiple exogenous views with minimal parameters. Empirically, the approach achieves competitive accuracy across four real-world datasets while reducing parameter counts substantially and providing interpretable insights into feature contributions. This yields robust, scalable forecasts suitable for practical deployment, with potential applicability to other time-series domains requiring efficient, explainable models.

Abstract

Time series forecasting is a critical task across domains such as energy, finance, and meteorology, where accurate predictions enable informed decision-making. While transformer-based and large-parameter models have recently achieved state-of-the-art results, their complexity can lead to overfitting and unstable forecasts, especially when older data points become less relevant. In this paper, we propose a lightweight multi-view approach to short-term load forecasting that leverages single-value embeddings and a scaled time-range input to capture temporally relevant features efficiently. We introduce an embedding dropout mechanism to prevent over-reliance on specific features and enhance interpretability. Our method achieves competitive performance with significantly fewer parameters, demonstrating robustness across multiple datasets, including scenarios with noisy or sparse data, and provides insights into the contributions of individual features to the forecast.

A Lightweight Multi-View Approach to Short-Term Load Forecasting

TL;DR

The paper addresses short-term load forecasting in settings with limited data and the risk of overfitting large models by introducing a lightweight multi-view framework. It combines single-value embeddings, a scaled time-range input of critical lags, and embedding dropout within a transformer-based encoder–decoder to fuse multiple exogenous views with minimal parameters. Empirically, the approach achieves competitive accuracy across four real-world datasets while reducing parameter counts substantially and providing interpretable insights into feature contributions. This yields robust, scalable forecasts suitable for practical deployment, with potential applicability to other time-series domains requiring efficient, explainable models.

Abstract

Time series forecasting is a critical task across domains such as energy, finance, and meteorology, where accurate predictions enable informed decision-making. While transformer-based and large-parameter models have recently achieved state-of-the-art results, their complexity can lead to overfitting and unstable forecasts, especially when older data points become less relevant. In this paper, we propose a lightweight multi-view approach to short-term load forecasting that leverages single-value embeddings and a scaled time-range input to capture temporally relevant features efficiently. We introduce an embedding dropout mechanism to prevent over-reliance on specific features and enhance interpretability. Our method achieves competitive performance with significantly fewer parameters, demonstrating robustness across multiple datasets, including scenarios with noisy or sparse data, and provides insights into the contributions of individual features to the forecast.
Paper Structure (15 sections, 11 equations, 3 figures, 8 tables)

This paper contains 15 sections, 11 equations, 3 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Related Work
  4. Preliminaries
  5. Transformers
  6. Encoder-Decoder models
  7. Embeddings
  8. Proposed Approach
  9. Experimental Settings
  10. Datasets
  11. Training and Testing
  12. Experimental Results
  13. Discussion
  14. Conclusion
  15. Technical Appendices and Supplementary Material

Figures (3)

  • Figure 1: Embeddings effects of long term, short term, temperature, weather and holidays related views on the IESO dataset for a 48 hours horizon using the SVD method. The combined effect is shown at the bottom right.
  • Figure 2: Embedded encoder representations where each cell indicates the weight for a discrete input category. Leftmost cells represent the lowest categorical values (e.g., Spring to Winter for season). The weighting vector $W$ reflects the encoder's relative emphasis on each view.
  • Figure 3: Forecasts from N-BEATS, SARIMAX, TFT, TiDE and our approach on a horizon of two days spanning from the beggining of January 1st 2013 to the end of January 2nd 2013 . Our approach is shown at the bottom right.