Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HOPS: High-order Polynomials with Self-supervised Dimension Reduction for Load Forecasting

Pengyang Song, Han Feng, Shreyashi Shukla, Jue Wang, Tao Hong

TL;DR

This paper tackles load forecasting with high-order multivariate polynomials by addressing the Curse of Dimensionality and overfitting through self-supervised, low-rank dimension reduction. It introduces HOPS, a framework that embeds high-order polynomial terms via a rank-$k$ linear map and solves the resulting regression efficiently with a PolyCG algorithm that exploits tensor structure and GPU acceleration. The authors establish theoretical guarantees for the embedding and provide an analytic SVD-based solution under Frobenius loss, while extensive ISO New England experiments show that HOPS achieves higher accuracy with fewer inputs than traditional models and even outperforms several advanced baselines. The work offers a practical, scalable approach that can be integrated with existing forecasting pipelines, enabling more accurate predictions with reduced computational demands.

Abstract

Load forecasting is a fundamental task in smart grid. Many techniques have been applied to developing load forecasting models. Due to the challenges such as the Curse of Dimensionality, overfitting, and limited computing resources, multivariate higher-order polynomial models have received limited attention in load forecasting, despite their desirable mathematical foundations and optimization properties. In this paper, we propose low rank approximation and self-supervised dimension reduction to address the aforementioned issues. To further improve computational efficiency, we also utilize a fast Conjugate Gradient based algorithm for the proposed polynomial models. Based on the load datasets from the ISO New England, the proposed method high-order polynomials with self-supervised dimension reduction (HOPS) demonstrates higher forecasting accuracy over several competitive models. Additionally, experimental results indicate that our approach alleviates redundant variable construction, achieving better forecasts with fewer input variables.

HOPS: High-order Polynomials with Self-supervised Dimension Reduction for Load Forecasting

TL;DR

This paper tackles load forecasting with high-order multivariate polynomials by addressing the Curse of Dimensionality and overfitting through self-supervised, low-rank dimension reduction. It introduces HOPS, a framework that embeds high-order polynomial terms via a rank- linear map and solves the resulting regression efficiently with a PolyCG algorithm that exploits tensor structure and GPU acceleration. The authors establish theoretical guarantees for the embedding and provide an analytic SVD-based solution under Frobenius loss, while extensive ISO New England experiments show that HOPS achieves higher accuracy with fewer inputs than traditional models and even outperforms several advanced baselines. The work offers a practical, scalable approach that can be integrated with existing forecasting pipelines, enabling more accurate predictions with reduced computational demands.

Abstract

Load forecasting is a fundamental task in smart grid. Many techniques have been applied to developing load forecasting models. Due to the challenges such as the Curse of Dimensionality, overfitting, and limited computing resources, multivariate higher-order polynomial models have received limited attention in load forecasting, despite their desirable mathematical foundations and optimization properties. In this paper, we propose low rank approximation and self-supervised dimension reduction to address the aforementioned issues. To further improve computational efficiency, we also utilize a fast Conjugate Gradient based algorithm for the proposed polynomial models. Based on the load datasets from the ISO New England, the proposed method high-order polynomials with self-supervised dimension reduction (HOPS) demonstrates higher forecasting accuracy over several competitive models. Additionally, experimental results indicate that our approach alleviates redundant variable construction, achieving better forecasts with fewer input variables.
Paper Structure (22 sections, 65 equations, 4 figures, 5 tables, 1 algorithm)

This paper contains 22 sections, 65 equations, 4 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Higher-order Polynomial Model and Self-supervised Dimension Reduction
  3. The Definition of $n$-dimensional $d$-order Polynomial
  4. Self-supervised Dimension Reduction
  5. Properties and the Solution
  6. A Fast Algorithm for Multivariate Higher-order Polynomial Model
  7. Algorithm Overview
  8. A Note on Algorithm \ref{['alg:1']}
  9. Numerical Experiment
  10. Datasets and Metrics
  11. Datasets
  12. Evaluation Metrics
  13. Implementation Details
  14. The Competitive Comparison Models
  15. Overall Performance with the Same or Simpler Inputs
  16. ...and 7 more sections

Figures (4)

  • Figure 1: The tensor product of $x$ and $x$, $x \otimes x$ and $x$.
  • Figure 2: An $n$-dimensional $3$-order polynomial.
  • Figure 3: An $n$-dimensional $3$-order polynomial with quadratic term embedding dimension $k_2$ and cubic term embedding dimension $k_3$.
  • Figure 4: The performance comparision of $G_1$, $H_1$, $H_2$, HOPS47, HOPS50 and HOPS59. $G_1$ and HOPS47, $H_1$ and HOPS50, $H_2$ and HOPS59 essentially share the same information, respectively. The forecasting accuracy of these models generally improves as the number of variables increases. However, when the input information keeps the same, our model consistently outperforms the existing models $G_1$, $H_1$ and $H_2$, respectively.

Theorems & Definitions (3)

  • Proof 6.1
  • Proof 6.2
  • Proof 6.3