Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MSHyper: Multi-Scale Hypergraph Transformer for Long-Range Time Series Forecasting

Zongjiang Shang, Ling Chen, Binqing Wu, Dongliang Cui

TL;DR

The paper tackles long-range time-series forecasting by modeling high-order interactions across multiple temporal scales. It introduces MSHyper, a Multi-Scale Hypergraph Transformer, which constructs a multi-scale hypergraph and a hyperedge graph and employs a tri-stage message passing mechanism to learn interaction strengths between patterns from different scales. Key contributions include the hypergraph and hyperedge graph construction (H-HGC) module, the tri-stage message passing (TMP) framework, and comprehensive experiments showing state-of-the-art performance across eight real-world datasets with robust long-horizon forecasting. The approach demonstrates that explicit high-order cross-scale interactions, captured via hypergraphs, yield substantial accuracy gains and better long-range robustness, with potential extensions to adaptive hypergraphs and neural architecture search.

Abstract

Demystifying interactions between temporal patterns of different scales is fundamental to precise long-range time series forecasting. However, previous works lack the ability to model high-order interactions. To promote more comprehensive pattern interaction modeling for long-range time series forecasting, we propose a Multi-Scale Hypergraph Transformer (MSHyper) framework. Specifically, a multi-scale hypergraph is introduced to provide foundations for modeling high-order pattern interactions. Then by treating hyperedges as nodes, we also build a hyperedge graph to enhance hypergraph modeling. In addition, a tri-stage message passing mechanism is introduced to aggregate pattern information and learn the interaction strength between temporal patterns of different scales. Extensive experiments on five real-world datasets demonstrate that MSHyper achieves state-of-the-art (SOTA) performance across various settings.

MSHyper: Multi-Scale Hypergraph Transformer for Long-Range Time Series Forecasting

TL;DR

The paper tackles long-range time-series forecasting by modeling high-order interactions across multiple temporal scales. It introduces MSHyper, a Multi-Scale Hypergraph Transformer, which constructs a multi-scale hypergraph and a hyperedge graph and employs a tri-stage message passing mechanism to learn interaction strengths between patterns from different scales. Key contributions include the hypergraph and hyperedge graph construction (H-HGC) module, the tri-stage message passing (TMP) framework, and comprehensive experiments showing state-of-the-art performance across eight real-world datasets with robust long-horizon forecasting. The approach demonstrates that explicit high-order cross-scale interactions, captured via hypergraphs, yield substantial accuracy gains and better long-range robustness, with potential extensions to adaptive hypergraphs and neural architecture search.

Abstract

Demystifying interactions between temporal patterns of different scales is fundamental to precise long-range time series forecasting. However, previous works lack the ability to model high-order interactions. To promote more comprehensive pattern interaction modeling for long-range time series forecasting, we propose a Multi-Scale Hypergraph Transformer (MSHyper) framework. Specifically, a multi-scale hypergraph is introduced to provide foundations for modeling high-order pattern interactions. Then by treating hyperedges as nodes, we also build a hyperedge graph to enhance hypergraph modeling. In addition, a tri-stage message passing mechanism is introduced to aggregate pattern information and learn the interaction strength between temporal patterns of different scales. Extensive experiments on five real-world datasets demonstrate that MSHyper achieves state-of-the-art (SOTA) performance across various settings.
Paper Structure (26 sections, 25 equations, 8 figures, 8 tables)

This paper contains 26 sections, 25 equations, 8 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methods for Time Series Forecasting
  4. Multi-Scale Transformers
  5. Hypergraph Neural Networks
  6. Preliminaries
  7. Hypergraph
  8. Problem Statement
  9. Methodology
  10. Framework
  11. Multi-Scale Feature Extraction Module
  12. Hypergraph and Hyperedge Graph Construction Module
  13. Hypergraph Construction
  14. Hyperedge Graph Construction
  15. Tri-Stage Message Passing Mechanism
  16. ...and 11 more sections

Figures (8)

  • Figure 1: The framework of MSHyper, which consists of three parts: (a) The HFE module, which maps the input sequence into hierarchical embeddings. (b) The H-HGC module, which provides foundations for modeling high-order interactions between temporal patterns by building the hypergraph and the hyperedge graph. (c) The TMP mechanism, which aggregates pattern information and learns the interaction strength between temporal patterns of different scales.
  • Figure 2: Hypergraph construction. (a), (b), and (c) represent the intra-scale hypergraph, inter-scale hypergraph, and mixed-scale hypergraph, respectively. In addition to the original connections, we also aggregate information from different ranges of neighbors using $k$-hop connections.
  • Figure 3: Hyperedge graph construction. (a) and (b) are the sequential relationship and association relationship, respectively. (c) and (d) are the constructed adjacency matrix based on the sequential relationship and association relationship, respectively.
  • Figure 4: Forecasting results of different models on Electricity dataset under the input-96-predict-96 setting. The black line represents the ground truth and the orange line represents the predicted results.
  • Figure 5: Forecasting results of different models on Weather dataset under the input-96-predict-96 setting. The black line represents the ground truth and the orange line represents the predicted results.
  • ...and 3 more figures