Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Advancing Long-Term Multi-Energy Load Forecasting with Patchformer: A Patch and Transformer-Based Approach

Qiuyi Hong, Fanlin Meng, Felipe Maldonado

TL;DR

Patchformer introduces a Patch Embedding–based encoder–decoder Transformer to address long-term multi-energy load forecasting in IMES. By treating each energy channel as a univariate sequence and segmenting it into patches, the model captures local semantics while learning inter-channel dependencies through a channel-independent embedding scheme. Extensive experiments on a new Multi-Energy dataset and six public benchmarks show superior multivariate and univariate forecasting performance, and reveal a positive link between past history length and accuracy. The approach offers practical benefits for robust, long-horizon energy planning and can be extended with explainability techniques to enhance trust in forecasting decisions.

Abstract

In the context of increasing demands for long-term multi-energy load forecasting in real-world applications, this paper introduces Patchformer, a novel model that integrates patch embedding with encoder-decoder Transformer-based architectures. To address the limitation in existing Transformer-based models, which struggle with intricate temporal patterns in long-term forecasting, Patchformer employs patch embedding, which predicts multivariate time-series data by separating it into multiple univariate data and segmenting each of them into multiple patches. This method effectively enhances the model's ability to capture local and global semantic dependencies. The numerical analysis shows that the Patchformer obtains overall better prediction accuracy in both multivariate and univariate long-term forecasting on the novel Multi-Energy dataset and other benchmark datasets. In addition, the positive effect of the interdependence among energy-related products on the performance of long-term time-series forecasting across Patchformer and other compared models is discovered, and the superiority of the Patchformer against other models is also demonstrated, which presents a significant advancement in handling the interdependence and complexities of long-term multi-energy forecasting. Lastly, Patchformer is illustrated as the only model that follows the positive correlation between model performance and the length of the past sequence, which states its ability to capture long-range past local semantic information.

Advancing Long-Term Multi-Energy Load Forecasting with Patchformer: A Patch and Transformer-Based Approach

TL;DR

Patchformer introduces a Patch Embedding–based encoder–decoder Transformer to address long-term multi-energy load forecasting in IMES. By treating each energy channel as a univariate sequence and segmenting it into patches, the model captures local semantics while learning inter-channel dependencies through a channel-independent embedding scheme. Extensive experiments on a new Multi-Energy dataset and six public benchmarks show superior multivariate and univariate forecasting performance, and reveal a positive link between past history length and accuracy. The approach offers practical benefits for robust, long-horizon energy planning and can be extended with explainability techniques to enhance trust in forecasting decisions.

Abstract

In the context of increasing demands for long-term multi-energy load forecasting in real-world applications, this paper introduces Patchformer, a novel model that integrates patch embedding with encoder-decoder Transformer-based architectures. To address the limitation in existing Transformer-based models, which struggle with intricate temporal patterns in long-term forecasting, Patchformer employs patch embedding, which predicts multivariate time-series data by separating it into multiple univariate data and segmenting each of them into multiple patches. This method effectively enhances the model's ability to capture local and global semantic dependencies. The numerical analysis shows that the Patchformer obtains overall better prediction accuracy in both multivariate and univariate long-term forecasting on the novel Multi-Energy dataset and other benchmark datasets. In addition, the positive effect of the interdependence among energy-related products on the performance of long-term time-series forecasting across Patchformer and other compared models is discovered, and the superiority of the Patchformer against other models is also demonstrated, which presents a significant advancement in handling the interdependence and complexities of long-term multi-energy forecasting. Lastly, Patchformer is illustrated as the only model that follows the positive correlation between model performance and the length of the past sequence, which states its ability to capture long-range past local semantic information.
Paper Structure (21 sections, 9 equations, 7 figures, 7 tables)

This paper contains 21 sections, 9 equations, 7 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Literature Review
  4. Contributions
  5. Paper Organisation
  6. Model Architecture
  7. Model Overview
  8. Patch Embedding
  9. Multi-Head Attention Block
  10. Layer Normalisation Block
  11. Feed Forward Block
  12. Encoder
  13. Decoder
  14. Flatten and Linear Head
  15. Numerical Analysis
  16. ...and 6 more sections

Figures (7)

  • Figure 1: Patchformer Architecture
  • Figure 2: Patch Embedding
  • Figure 3: Scaled Dot-Product Attention Block (left) and Multi-Head Attention Block (right), including $H$ attention layers (heads).
  • Figure 4: Visualisation of 192 step forecasting on Multi-Energy, Exchange, Weather, ETTh1 and ETTm1 datasets with the past time sequence = 96.
  • Figure 5: Visualisation of 192 step forecasting on Electricity, Gas and GHG in Multi-Energy dataset with the past sequence length = 336.
  • ...and 2 more figures