Learning Global and Local Features of Power Load Series Through Transformer and 2D-CNN: An Image-based Multi-step Forecasting Approach Incorporating Phase Space Reconstruction

TL;DR

This work tackles non-stationary power load forecasting by embedding power load series in phase space through PSR and transforming the resulting trajectories into grayscale images. It introduces PSR-GALIEN, a two-branch deep learning architecture that uses a Transformer Encoder for global pattern extraction and a 2D-CNN for local pattern extraction, with an MLP predictor to model cross-feature correlations in an end-to-end framework for multi-step ahead forecasting. The authors establish an equivalence between PSR and patch segmentation as data structures and demonstrate superior accuracy over six baselines across five real-world datasets in both intra-day and day-ahead tasks, while also offering RAM and heatmap-based interpretability of the learned features. The approach provides a scalable, interpretable alternative to exogenous-feature-heavy models, with practical implications for robust short- to mid-term power load forecasting. Overall, PSR-GALIEN advances phase-space-informed forecasting by effectively leveraging global and local features in image-like representations of reconstructed attractors, delivering improved accuracy and interpretability for real-world energy management.

Abstract

As modern power systems continue to evolve, accurate power load forecasting remains a critical issue in energy management. The phase space reconstruction method can effectively retain the inner chaotic property of power load from a system dynamics perspective and thus is a promising knowledge-based preprocessing method for short-term forecasting. In order to fully utilize the capability of PSR method to model the non-stationary characteristics within power load, and to solve the problem of the difficulty in applying traditional PSR prediction methods to form a general multi-step forecasting scheme, this study proposes a novel multi-step forecasting approach by delicately integrating the PSR with neural networks to establish an end-to-end learning system. Firstly, the useful features in the phase trajectory are discussed in detail. Through mathematical derivation, the equivalent characterization of the PSR and another time series preprocessing method, patch segmentation, is demonstrated for the first time. Based on this knowledge, an image-based modeling perspective is introduced. Subsequently, a novel deep learning model, namely PSR-GALIEN, is designed, in which the Transformer Encoder and 2D-CNN are employed for the extraction of the global and local patterns in the image, and a MLP-based predictor is used for the efficient correlation modeling. Then, extensive experiments are conducted on five real-world benchmark datasets to verify the effectiveness of the PSR-GALIEN. The results show that, compared with six state-of-the-art deep learning models, the forecasting performance of PSR-GALIEN consistently surpasses these baselines, achieving superior accuracy in both intra-day and day-ahead forecasting scenarios. At the same time, the attributions of its forecasting results can be explained through the visualization-based method, which significantly increases the interpretability.

Figures (11)

• Figure 1: Image-based feature extraction and forecasting framework incorporating the PSR preprocessing method
• Figure 2: Architecture of the proposed PSR-GALIEN model, where $n_{\rm{heads}}$,$e_{\rm{layers}}$, $d_{\rm{model}}$, and $d_{\rm{ff}}$ are hyperparameters representing the number of heads in the multi-head attention layer of Transformer Encoder,the number of implemented Encoder layers, the number of hidden neurons in the feature extraction network and in the MLP predictor, respectively.
• Figure 3: Visualization of the five datasets
• Figure 4: Forecasting results of each model with fixed input length (L=192) and varying forecasting length (P) on the Elia-2022, City-1, Area-1, Area-2 and Australia datasets, the prediction horizons in each dataset are 3h, 6h, 12h, and 24h in advance.
• Figure 5: Forecasting results of each model with fixed out length (P=96) and varying input length (L) on the City-1, Area-1, Area-2 datasets.